http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1276991242.1276992271.20801.gz Rev3 Fedora 12 mozilla-central opt test xpcshell on 2010/06/19 16:47:22 s: talos-r3-fed-048 TEST-UNEXPECTED-FAIL | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js | test failed (with xpcshell return code: 0), see following log: >>>>>>> TEST-INFO | (xpcshell/head.js) | test 1 pending arg[0] = -g arg[1] = /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/firefox arg[2] = -f arg[3] = /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/crasher_subprocess_head.js arg[4] = -f arg[5] = /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/crasher_subprocess_tail.js TEST-PASS | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/head_crashreporter.js | [do_crash : 65] 256 != 0 TEST-PASS | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/head_crashreporter.js | [do_crash : 84] true == true TEST-PASS | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js | [anonymous : 10] true == true TEST-PASS | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js | [anonymous : 11] true == true TEST-PASS | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js | [anonymous : 12] true == true TEST-UNEXPECTED-FAIL | /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js | false == true - See following stack: JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/head.js :: do_throw :: line 257 JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/head.js :: do_check_eq :: line 287 JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/head.js :: do_check_true :: line 299 JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js :: anonymous :: line 13 JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/head_crashreporter.js :: do_crash :: line 88 JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/tests/crashreporter_test/unit/test_crashreporter_crash.js :: run_test :: line 9 JS frame :: /home/cltbld/talos-slave/mozilla-central-fedora-opt-u-xpcshell/build/xpcshell/head.js :: _execute_test :: line 151 JS frame :: -e :: <TOP_LEVEL> :: line 1 TEST-INFO | (xpcshell/head.js) | exiting test <<<<<<<

I'm getting this a lot with bug 568619 branch, and I can reproduce it very consistently on x86-linux if I don't debug it ;-) AnnotateCrashReport(StartupTime, 1277824175) crashReporterAPIData=StartupTime=1277824175 ExceptionHandler::GenerateDump API string: StartupTime=1277824175 MinidumpCallback, crashReporterAPIData<8397332>: open So sometime during generation of the minidump we trash crashReporterAPIData. Attaching a debugger, or adding a busy-sleep to the process initialization seems to fix it.

Holy Moses. We're lazily trying to resolve a symbol (open in this case) and the dynamic loader is trashing memory. I'm not sure *why* it would trash memory at this point, but I think the simple answer here is to force the symbols we need to be early-loaded.

Comment on attachment 454878 [details] [diff] [review] Resolve open/write early, rev. 1 Yuck, but ok.

bsmedberg: did you land your other patch and not this, thus making it perma-orange?

No, I landed http://hg.mozilla.org/mozilla-central/rev/99ac7d16e904 with the other one and it didn't help. I have now disabled the crashreporter tests on Linux-x86 until I can experiment with using the libc-replacement stuff.

The plugin crash-submission tests were still failing after the band-aid; pushed http://hg.mozilla.org/mozilla-central/rev/567178446a35 to disable those too.

This backs out the previous patch and the test disabling, and uses the Breakpad wrappers for Linux syscalls, as well as the trivial Breakpad libc helpers. The tests pass on my x86-64 build, and I'm doing a quick OS X build to ensure that I didn't break that, and I'll also push-to-try to check that this fixes the test bustage.

Comment on attachment 455674 [details] [diff] [review] Use libc/syscall wrappers on Linux Works on OS X, pushing to try to see if it fixes the test bustage.

Comment on attachment 455674 [details] [diff] [review] Use libc/syscall wrappers on Linux Tests still fail on try.

Since this bug is now getting spammed with the x86-64 test failures, I think we should land this patch. This fixes the test failure on my x86-64 machine, but last I tried it it didn't fix x86.

I pushed this to m-c with rs=khuey: http://hg.mozilla.org/mozilla-central/rev/43e26f5fad26

May have found related bug: 691632

Excitingly, we haven't been running these tests at all recently: bug 733501.