The following testcase crashes on mozilla-central revision 0876695d1abd (build with --enable-optimize --enable-posix-nspr-emulation --enable-valgrind --enable-gczeal --disable-tests --enable-debug, run with --fuzzing-safe --thread-count=2): var lfcode = new Array(); lfcode.push(` function TestCase(e) { this.expect = e; } function writeHeaderToLog() {} var SECTION = "15.5.4.7-1"; var TITLE = "String.protoype.lastIndexOf"; writeHeaderToLog(); var j = 0; for (k = 0, i = 0x0021; i < 0x007e; i++, j++, k++) new TestCase("x" - 1); LastIndexOf(); function LastIndexOf() { if (isNaN(n)) {} } `); lfcode.push(` oomAfterAllocations(50); writeHeaderToLog(SECTION + " " + TITLE); var expect = "Passed"; try { eval("this = true"); } catch (e) { result = expect; exception = e.toString(0, 0); } new TestCase(); `); while (true) { var file = lfcode.shift(); loadFile(file) } function loadFile(lfVarx) { try { if (lfVarx.substr(-3) != ".js" && lfVarx.length != 1) { evaluate(lfVarx); } } catch (lfVare) {} } Backtrace: Program received signal SIGSEGV, Segmentation fault. 0x0000000000641689 in js::ObjectGroup::maybeGetProperty (this=0x7ffff7e5db50, id=...) at js/src/vm/TypeInference-inl.h:1042 #0 0x0000000000641689 in js::ObjectGroup::maybeGetProperty (this=0x7ffff7e5db50, id=...) at js/src/vm/TypeInference-inl.h:1042 #1 0x00000000007c10c3 in js::UnboxedLayout::makeNativeGroup (cx=cx@entry=0x7ffff6907000, group=0x7ffff7e5d850) at js/src/vm/UnboxedObject.cpp:523 #2 0x00000000007bdb5f in js::UnboxedPlainObject::convertToNative (cx=cx@entry=0x7ffff6907000, obj=0x7ffff4800d60) at js/src/vm/UnboxedObject.cpp:547 #3 0x00000000007c2dc3 in js::UnboxedPlainObject::obj_setProperty (cx=0x7ffff6907000, obj=..., id=..., v=..., receiver=..., result=...) at js/src/vm/UnboxedObject.cpp:816 #4 0x0000000000b4c9c4 in JSObject::nonNativeSetProperty (cx=cx@entry=0x7ffff6907000, obj=..., id=..., v=..., receiver=..., result=...) at js/src/jsobj.cpp:1049 #5 0x0000000000706188 in js::SetProperty (cx=cx@entry=0x7ffff6907000, obj=..., obj@entry=..., id=..., id@entry=..., v=..., receiver=..., receiver@entry=..., result=...) at js/src/vm/NativeObject.h:1433 #6 0x000000000070652a in js::PutProperty (cx=0x7ffff6907000, obj=..., id=..., v=..., strict=<optimized out>) at js/src/jsobj.h:907 #7 0x00000000008e0485 in js::jit::DoSetPropFallback (cx=0x7ffff6907000, frame=<optimized out>, stub_=<optimized out>, lhs=..., rhs=..., res=...) at js/src/jit/BaselineIC.cpp:8697 #8 0x00007ffff7feed4f in ?? () [...] #32 0x0000000000000000 in ?? () rax 0x0 0 rbx 0x7ffff7e7e1d8 140737352557016 rcx 0x7ffff6ca53cd 140737333842893 rdx 0x0 0 rsi 0x7ffff6f7a9d0 140737336814032 rdi 0x7ffff6f791c0 140737336807872 rbp 0x7fffffffb090 140737488334992 rsp 0x7fffffffb060 140737488334944 r8 0x7ffff7fe0780 140737354008448 r9 0x6372732f736a2f6c 7165916604736876396 r10 0x7fffffffae20 140737488334368 r11 0x7ffff6c27960 140737333328224 r12 0x7ffff7e5db50 140737352424272 r13 0x7ff0000 134152192 r14 0x7ffff7e5db50 140737352424272 r15 0x7ffff7e7e1d8 140737352557016 rip 0x641689 <js::ObjectGroup::maybeGetProperty(jsid)+841> => 0x641689 <js::ObjectGroup::maybeGetProperty(jsid)+841>: movl $0x412,0x0 0x641694 <js::ObjectGroup::maybeGetProperty(jsid)+852>: callq 0x499bc0 <abort()>

The following assert in ObjectGroup::maybeGetProperty() is failing: MOZ_ASSERT(!unknownProperties()); ObjectGroup::getProprty() marks the group as having unknown properties if there is an allocation failure. This is called from AddTypePropertyId() via PropagatePropertyTypes() in UnboxedLayout::makeNativeGroup(). So an OOM in while propagating property trees causes this assertion to fail.

Patch to fail PropagatePropertyTypes() if the group ends up getting marked as having unknown properties.

Comment on attachment 8650535 [details] [diff] [review] bug1195452-PropagatePropertyTypes Review of attachment 8650535 [details] [diff] [review]: ----------------------------------------------------------------- This seems like a roundabout way of avoiding the failure in makeNativeGroup. It also has the problem that AddTypePropertyId does not report an exception on OOM (since it is infallible) so we'll end up returning false from PropagatePropertyTypes with no pending exception. I think a better fix would be for makeNativeGroup to be robust against the group being given unknown properties by PropagatePropertyTypes.

OK cool, patch updated.

Comment on attachment 8651121 [details] [diff] [review] bug1195452-PropagatePropertyTypes v2 Review of attachment 8651121 [details] [diff] [review]: ----------------------------------------------------------------- Thanks!

https://hg.mozilla.org/integration/mozilla-inbound/rev/a7fb70a151a1f562412c74e0d424f8a15cf48b6b Bug 1195452 - Make UnboxedLayout::makeNativeGroup() handle OOM when propagating property types r=bhackett