Most inference code doesn't handle or propagate OOM conditions when constructing type information or doing analysis. Conceptually, once bug 613221 is done it is OK to abort analysis of a script (which will not then be compiled), but any operation updating the core type information (making a type object, adding to the type set of a variable/property) must cause the corresponding JS operation to fail on OOM. The goal here is to preserve the invariant that the core type information overapproximates the feasible types in the compartment.

This fixes OOM handling for inference code and code which uses inference. It also removes --enable-type-inference and JS_TYPE_INFERENCE, instead making inference a runtime switch ('-n' in the shell, eventually javascript.options.typeinference in the browser). Inference is enabled per-compartment, at the time when the compartment is created. http://hg.mozilla.org/projects/jaegermonkey/rev/386e9cffdcd7

Comment 0 is too optimistic --- for most allocations during inference, if the allocation fails and we unwind the stack we still leave the analysis in an inconsistent state (e.g. type constraints not fully propagated) which we can't correctly recover from the next time someone tries to update the type information. So we want to just disable inference on compartments after an OOM. Before the interpoline this was problematic, as disabling inference required recompilation and recompilation could fail (especially if there was just an OOM). Now, however, we can discard jitcode infallibly by redirecting those frames to the interpreter, which makes handling OOM during analysis much simpler. http://hg.mozilla.org/projects/jaegermonkey/rev/9aeb58c8c43f