reproduce step

download the newset spidermonkey source code and build with

ac_add_options --enable-application=js
ac_add_options --enable-optimize
ac_add_options --enable-debug
ac_add_options --disable-shared-js
ac_add_options --enable-fuzzing
ac_add_options --enable-gczeal

Then run the following poc

Poc

const o0 = {
};
const v1 = [791.275797116544,0.952341230300292];
const o3 = {
    "newCompartment": true,
};
const t6 = newGlobal(o3).Debugger;
const t7 = t6(o0).memory;
t7.trackingAllocationSites = v1;
function f10() {
    new Float64Array(Float64Array);
    return v1;
}
Object.prototype.__defineGetter__(0, f10);
(0)[0]();

part of back trace

==12589==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address 0x000000000000 (pc 0x56318b20ea65 bp 0x7fffa86256d0 sp 0x7fffa86252c0 T12589)
==12589==The signal is caused by a WRITE memory access.
==12589==Hint: address points to the zero page.
    #0 0x56318b20ea65 in MOZ_Crash(char const*, int, char const*) /home/builder/firefox/obj-debugbuild/dist/include/mozilla/Assertions.h:261:3
    #1 0x56318b20ea65 in js::AutoEnterOOMUnsafeRegion::crash(char const*) /home/builder/firefox/js/src/vm/JSContext.cpp:1304:3
    #2 0x56318b35f185 in js::SavedStacks::MetadataBuilder::build(JSContext*, JS::Handle<JSObject*>, js::AutoEnterOOMUnsafeRegion&) const /home/builder/firefox/js/src/vm/SavedStacks.cpp
    #3 0x56318b345862 in JS::Realm::setNewObjectMetadata(JSContext*, JS::Handle<JSObject*>) /home/builder/firefox/js/src/vm/Realm.cpp:401:39
    #4 0x56318afc7a3d in js::NativeObject* js::SetNewObjectMetadata<js::NativeObject>(JSContext*, js::NativeObject*) /home/builder/firefox/js/src/vm/JSObject-inl.h:199:18
    #5 0x56318b23fdac in NewObject(JSContext*, JSClass const*, JS::Handle<js::TaggedProto>, js::gc::AllocKind, js::NewObjectKind) /home/builder/firefox/js/src/vm/JSObject.cpp:773:23

VERSION

JavaScript-C115.0a1
commit 2258e7d928427bf4a429075a7c2cba9ca89246e7

Thanks for the report. However, safe OOM crashes are not a security issue.

Jan, do you understand from where this OOM happens?
Reading the test case I am clueless about what is happening here.

Below is a simplified test case.

What's happening is that the debugger enables the object metadata callback. We end up in Debugger::appendAllocationSite where we call Compartment::wrap, which calls Compartment::getNonWrapperObjectForCurrentCompartment. There we do an overrecursion check with the system-limit (checkSystem) and this fails and throws an exception. We return to the metadata builder code where we crash.

It's a little weird that we crash with "[unhandlable oom]" for an overrecursion exception, but other than that this is all working as expected. For fuzzing it's best to ignore these "[unhandlable oom]" crashes, it's why this mechanism was added.

function f() {
    try {
        f();
    } catch (e) {
        new Int32Array(0);
    }
}
const g = newGlobal({newCompartment: true});
g.Debugger({}).memory.trackingAllocationSites = true;
f();