Since I've started fuzzing on real ARM64 hardware, I've been seeing GC crashes that I only see there and not on any other JS fuzzing target. Here is an example trace from mozilla-central revision 5ec55f7a95f9: Backtrace: Program terminated with signal SIGSEGV, Segmentation fault. #0 js::gc::ChunkBitmap::markIfUnmarked (color=<optimized out>, cell=<optimized out>, this=<optimized out>) at js/src/gc/Heap.h:629 #1 js::gc::TenuredCell::markIfUnmarked (color=<optimized out>, this=<optimized out>) at js/src/gc/Cell.h:305 #2 js::GCMarker::mark<JSObject> (thing=0x494949494949, this=0xffffa681a090) at js/src/gc/Marking.cpp:1016 #3 js::GCMarker::processMarkStackTop (this=this@entry=0xffffa681a090, budget=...) at js/src/gc/Marking.cpp:1782 #4 0x0000000000b56a34 in js::GCMarker::drainMarkStack (this=this@entry=0xffffa681a090, budget=...) at js/src/gc/Marking.cpp:1631 #5 0x0000000000b0dab8 in js::gc::GCRuntime::drainMarkStack (this=0xffffa68194b8, sliceBudget=..., phase=js::gcstats::PhaseKind::MARK) at js/src/gc/GC.cpp:5836 #6 0x0000000000b298a4 in js::gc::GCRuntime::incrementalCollectSlice (this=this@entry=0xffffa68194b8, budget=..., reason=reason@entry=JS::gcreason::DEBUG_GC, session=...) at js/src/gc/GC.cpp:7032 #7 0x0000000000b2a4ac in js::gc::GCRuntime::gcCycle (this=this@entry=0xffffa68194b8, nonincrementalByAPI=nonincrementalByAPI@entry=true, budget=..., reason=reason@entry=JS::gcreason::DEBUG_GC) at js/src/gc/GC.cpp:7394 #8 0x0000000000b2a8c0 in js::gc::GCRuntime::collect (this=this@entry=0xffffa68194b8, nonincrementalByAPI=nonincrementalByAPI@entry=true, budget=..., reason=reason@entry=JS::gcreason::DEBUG_GC) at js/src/gc/GC.cpp:7537 #9 0x0000000000b2b7d0 in js::gc::GCRuntime::gc (this=this@entry=0xffffa68194b8, reason=reason@entry=JS::gcreason::DEBUG_GC, gckind=GC_SHRINK) at js/src/gc/GC.cpp:7607 #10 0x0000000000b2b8c8 in js::gc::GCRuntime::runDebugGC (this=0xffffa68194b8) at js/src/gc/GC.cpp:8165 #11 0x0000000000b2bee4 in js::gc::GCRuntime::gcIfNeededAtAllocation (this=0xffffa68194b8, cx=0xffffa6815000) at js/src/gc/Allocator.cpp:310 #12 0x0000000000b456cc in js::gc::GCRuntime::checkAllocatorState<(js::AllowGC)1> (this=<optimized out>, cx=0xffffa6815000, kind=<optimized out>) at js/src/gc/Allocator.cpp:271 #13 0x0000000000b45b14 in js::AllocateString<JSString, (js::AllowGC)1> (cx=cx@entry=0xffffa6815000, heap=heap@entry=js::gc::DefaultHeap) at js/src/gc/Allocator.cpp:179 #14 0x00000000009a43f4 in js::Allocate<JSRope, (js::AllowGC)1> (heap=js::gc::DefaultHeap, cx=0xffffa6815000) at js/src/gc/Allocator.h:47 #15 JSRope::new_<(js::AllowGC)1> (length=45248, right=..., left=..., cx=0xffffa6815000) at js/src/vm/StringType-inl.h:125 #16 js::ConcatStrings<(js::AllowGC)1> (cx=0xffffa6815000, left=..., right=...) at js/src/vm/StringType.cpp:696 #17 0x000000000077976c in js::jit::DoConcatStrings (cx=<optimized out>, lhs=..., rhs=..., res=...) at js/src/jit/SharedIC.cpp:930 #18 0x00001f28a9ffcf20 in ?? () Backtrace stopped: previous frame inner to this frame (corrupt stack?) x0 0x12929 76073 x1 0x494fc0a0 80579111403680 x2 0xa65d07a0 281473472858016 x3 0x29 41 x4 0x4a4 1188 x5 0x1 1 x6 0x689 1673 x7 0x2100 8448 x8 0x2a3 675 x9 0x11111111 1229782938247303441 x10 0x13af4 80628 x11 0x1685083a 377817146 x12 0x18 24 x13 0xa53bdc7a -1522803590 x14 0x2b000000 14617312028000256 x15 0x0 16777216000000000 x16 0xf202d8 15860440 x17 0xa7636cf8 281473490054392 x18 0x1 1 x19 0x49494949 80579110979913 x20 0xf1f000 15855616 x21 0x1fff7 131063 x22 0xfdceb1d0 281474939924944 x23 0xfdceb1c8 281474939924936 x24 0xfffe8 1048552 x25 0xfdceb510 281474939925776 x26 0xa681a090 281473475256464 x27 0x1fff6 131062 x28 0xa68194b8 281473475253432 x29 0xfdceb150 281474939924816 x30 0xb56a34 11889204 sp 0xfdceb150 281474939924816 pc 0xb65bfc <js::GCMarker::processMarkStackTop(js::SliceBudget&)+732> cpsr 0x20000000 536870912 fpcsr void fpcr 0x0 0 => 0xb65bfc <js::GCMarker::processMarkStackTop(js::SliceBudget&)+732>: ldr x6, [x1,x4,lsl #3] 0xb65c00 <js::GCMarker::processMarkStackTop(js::SliceBudget&)+736>: lsl x3, x5, x3 I discussed this already with :sfink a bit and the fact that it only reproduces on ARM64 might indicate that it is related to a locking problem (race) that doesn't manifest on x86. Unfortunately, so far I have no tests that would reproduce the issue. All of the tests the fuzzer comes up with don't reproduce, which supports the hypothesis that this is some form of race. The crash is happening on a thing=0x494949494949, indicating that a compacting GC is involved in this.

Putting a needinfo on Jon, maybe he has an idea what could be the cause.

That's interesting. I'll think about how this could happen.

I ran the tests under TSAN with GC zeal enabled to see if that showed up any races, but it didn't find anything related to this. This was on x64 of course - I don't have any ARM64 hardware.