padenot noticed that he was getting crashes with deep stacks when using std::thread: (gdb) up 100000 #100060 0x000000000041824b in std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>, void (*)()) () (gdb) up #100061 0x00007ffff76ececd in std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (gdb) up #100062 0x000000000041824b in std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>, void (*)()) () (gdb) up #100063 0x00007ffff76ececd in std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (gdb) up #100064 0x000000000041824b in std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>, void (*)()) () (gdb) up #100065 0x00007ffff76ececd in std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 ...and this continues on for quite some time. The problem here is that we override _M_start_thread(shared_ptr, void (*)()): https://searchfox.org/mozilla-central/source/build/unix/stdc++compat/stdc++compat.cpp#128 But the libstdc++ 6 sources use that symbol to actually start the thread: https://github.com/gcc-mirror/gcc/blob/gcc-6-branch/libstdc%2B%2B-v3/src/c%2B%2B11/thread.cc#L184-L197 and our override calls _M_start_thread(shared_ptr), which just calls the other overload: https://github.com/gcc-mirror/gcc/blob/gcc-6-branch/libstdc%2B%2B-v3/src/c%2B%2B11/thread.cc#L170-L182 So we get the infinite recursion seen in padenot's debugging session. The setup is the same for the libstdc++ 5 sources: https://github.com/gcc-mirror/gcc/blob/gcc-5-branch/libstdc%2B%2B-v3/src/c%2B%2B11/thread.cc#L129-L155 The libstdc++ 4.9 sources, however, start the thread from the _M_start_thread(shared_ptr) overload: https://github.com/gcc-mirror/gcc/blob/gcc-4_9-branch/libstdc++-v3/src/c++11/thread.cc#L129-L148 This also means that any other std::thread usages are busted, but at least according to crash-stats, padenot is the first person to create std::thread in anger. This could silently bite us in other ways if webrtc or skia or something starts using std::thread; presumably our test suite would catch such things, but it would also not totally surprise me if such things made it into Nightly, at least, with minimal testing. I think this means we have a problem, because we need to have the symbol override so we don't depend on a newer libstdc++, but we need to figure out which libstdc++ version we're using at runtime so we can call the right symbol: - the shared_ptr overload if we're using libstdc++ 4.9; or - the shared_ptr, funcptr overload if we're using something different; or - reimplement more bits using private routines (ick?). And ideally GCC won't decide to switch back in a future version of libstdc++. Marking statuses appropriately; we could track this for 62, I guess?

I have a patch for this, just need to convince everything to link libdl appropriately.

Tracking to keep an eye on this for 62.

Should we track this for 61? We may only not catch this on automation because of the libstdc++ version.

A simpler fix is to stop exporting stl symbols from libxul.

Comment on attachment 8982390 [details] Bug 1464084 - Don't export std::thread::_M_start_thread symbols with --enable-stdcxx-compat. https://reviewboard.mozilla.org/r/248328/#review254570 This ought to work. Have you tested with padenot's test re-enabled on try?

Comment on attachment 8982390 [details] Bug 1464084 - Don't export std::thread::_M_start_thread symbols with --enable-stdcxx-compat. https://reviewboard.mozilla.org/r/248328/#review254816 ::: python/mozbuild/mozbuild/action/generate_symbols_file.py:75 (Diff revision 3) > # A linker version script is generated for GNU LD that looks like the > # following: > # { Update this comment to reflect that we're sticking the library name in here now?

Pushed by mh@glandium.org: https://hg.mozilla.org/integration/autoland/rev/838c0ab9cbdb Don't export libstdc++ symbols with --enable-stdcxx-compat. r=froydnj

Backed out for Linux static bustage. backout: https://hg.mozilla.org/integration/autoland/rev/74dffd02a249f90777df32ca01579e6c5d3e6a3f push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=838c0ab9cbdbe85b0f7997ec48289d0a7fc12360&selectedJob=181443402 failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=181443400&repo=autoland&lineNumber=3904 [task 2018-06-01T22:43:34.378Z] 22:43:34 INFO - /builds/worker/workspace/build/src/sccache2/sccache /builds/worker/workspace/build/src/clang/bin/clang++ -std=gnu++14 -o TestNoRefcountedInsideLambdas.o -c -DNDEBUG=1 -DTRIMMED=1 -I/builds/worker/workspace/build/src/build/clang-plugin/tests -I/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -fPIC -DMOZILLA_CLIENT -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Qunused-arguments -D_GLIBCXX_USE_CXX11_ABI=0 -fno-sized-deallocation -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -fsyntax-only -Xclang -verify -ferror-limit=0 -Wno-invalid-noreturn -g -Xclang -load -Xclang /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O2 -fno-omit-frame-pointer -Werror -MD -MP -MF .deps/TestNoRefcountedInsideLambdas.o.pp /builds/worker/workspace/build/src/build/clang-plugin/tests/TestNoRefcountedInsideLambdas.cpp [task 2018-06-01T22:43:34.378Z] 22:43:34 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' [task 2018-06-01T22:43:34.486Z] 22:43:34 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' [task 2018-06-01T22:43:34.486Z] 22:43:34 INFO - /builds/worker/workspace/build/src/sccache2/sccache /builds/worker/workspace/build/src/clang/bin/clang++ -std=gnu++14 -o TestNonMemMovable.o -c -DNDEBUG=1 -DTRIMMED=1 -I/builds/worker/workspace/build/src/build/clang-plugin/tests -I/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -fPIC -DMOZILLA_CLIENT -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Qunused-arguments -D_GLIBCXX_USE_CXX11_ABI=0 -fno-sized-deallocation -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -fsyntax-only -Xclang -verify -ferror-limit=0 -Wno-invalid-noreturn -g -Xclang -load -Xclang /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O2 -fno-omit-frame-pointer -Werror -MD -MP -MF .deps/TestNonMemMovable.o.pp /builds/worker/workspace/build/src/build/clang-plugin/tests/TestNonMemMovable.cpp [task 2018-06-01T22:43:34.486Z] 22:43:34 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' [task 2018-06-01T23:23:34.502Z] 23:23:34 INFO - Automation Error: mozprocess timed out after 2400 seconds running ['/usr/bin/python2.7', 'mach', '--log-no-times', 'build', '-v'] [task 2018-06-01T23:23:34.514Z] 23:23:34 ERROR - timed out after 2400 seconds of no output [task 2018-06-01T23:23:34.514Z] 23:23:34 ERROR - Return code: -15 [task 2018-06-01T23:23:34.514Z] 23:23:34 WARNING - setting return code to 2 [task 2018-06-01T23:23:34.514Z] 23:23:34 FATAL - 'mach build -v' did not run successfully. Please check log for errors. [task 2018-06-01T23:23:34.514Z] 23:23:34 FATAL - Running post_fatal callback... [task 2018-06-01T23:23:34.515Z] 23:23:34 FATAL - Exiting -1 [task 2018-06-01T23:23:34.515Z] 23:23:34 INFO - [mozharness: 2018-06-01 23:23:34.515167Z] Finished build step (failed) [task 2018-06-01T23:23:34.515Z] 23:23:34 INFO - Running post-run listener: _summarize [task 2018-06-01T23:23:34.515Z] 23:23:34 ERROR - # TBPL FAILURE # [task 2018-06-01T23:23:34.515Z] 23:23:34 INFO - [mozharness: 2018-06-01 23:23:34.515398Z] FxDesktopBuild summary: [task 2018-06-01T23:23:34.515Z] 23:23:34 ERROR - # TBPL FAILURE #

Huh, looks like the clang-plugin didn't like this *** glibc detected *** /builds/worker/workspace/build/src/clang/bin/clang-5.0: free(): invalid pointer: 0x0000000003c8bf80 *** ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x75bb6)[0x7fb8f696fbb6] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x6c)[0x7fb8f697495c] /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so(_ZN26DanglingOnTemporaryChecker5checkERKN5clang12ast_matchers11MatchFinder11MatchResultE+0x8c1)[0x7fb8f6608fa1] /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29a1bdf] /builds/worker/workspace/build/src/clang/bin/clang-5.0(_ZN5clang12ast_matchers8internal21BoundNodesTreeBuilder12visitMatchesEPNS2_7VisitorE+0x172)[0x29b76d2] [task 2018-06-01T22:43:34.054Z] 22:43:34 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' [task 2018-06-01T22:43:34.054Z] 22:43:34 INFO - /builds/worker/workspace/build/src/sccache2/sccache /builds/worker/workspace/build/src/clang/bin/clang -std=gnu99 -o TestNANTestingExprC.o -c -DNDEBUG=1 -DTRIMMED=1 -I/builds/worker/workspace/build/src/build/clang-plugin/tests -I/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -fPIC -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -DMOZILLA_CLIENT -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-strict-aliasing -ffunction-sections -fdata-sections -fno-math-errno -pthread -pipe -fsyntax-only -Xclang -verify -ferror-limit=0 -std=c11 -Wno-invalid-noreturn -g -Xclang -load -Xclang /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O2 -fno-omit-frame-pointer -Werror -MD -MP -MF .deps/TestNANTestingExprC.o.pp /builds/worker/workspace/build/src/build/clang-plugin/tests/TestNANTestingExprC.c [task 2018-06-01T22:43:34.054Z] 22:43:34 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' [task 2018-06-01T22:43:34.055Z] 22:43:34 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29a1a8b] [task 2018-06-01T22:43:34.055Z] 22:43:34 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' [task 2018-06-01T22:43:34.055Z] 22:43:34 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' [task 2018-06-01T22:43:34.055Z] 22:43:34 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29adc2a] /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29a7cb7] /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29a4d05] [task 2018-06-01T22:43:34.056Z] 22:43:34 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29b0b46] [task 2018-06-01T22:43:34.058Z] 22:43:34 INFO - /builds/worker/workspace/build/src/sccache2/sccache /builds/worker/workspace/build/src/clang/bin/clang++ -std=gnu++14 -o TestExplicitOperatorBool.o -c -DNDEBUG=1 -DTRIMMED=1 -I/builds/worker/workspace/build/src/build/clang-plugin/tests -I/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -fPIC -DMOZILLA_CLIENT -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Qunused-arguments -D_GLIBCXX_USE_CXX11_ABI=0 -fno-sized-deallocation -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -fsyntax-only -Xclang -verify -ferror-limit=0 -Wno-invalid-noreturn -g -Xclang -load -Xclang /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O2 -fno-omit-frame-pointer -Werror -MD -MP -MF .deps/TestExplicitOperatorBool.o.pp /builds/worker/workspace/build/src/build/clang-plugin/tests/TestExplicitOperatorBool.cpp [task 2018-06-01T22:43:34.058Z] 22:43:34 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' [task 2018-06-01T22:43:34.058Z] 22:43:34 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29a7cb7] /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29aa62c] /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29a7cb7] /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29b4f4a] /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29a1f11] /builds/worker/workspace/build/src/clang/bin/clang-5.0[task 2018-06-01T22:43:34.062Z] 22:43:34 INFO - /builds/worker/workspace/build/src/sccache2/sccache /builds/worker/workspace/build/src/clang/bin/clang++ -std=gnu++14 -o TestBadImplicitConversionCtor.o -c -DNDEBUG=1 -DTRIMMED=1 -I/builds/worker/workspace/build/src/build/clang-plugin/tests -I/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -fPIC -DMOZILLA_CLIENT -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Qunused-arguments -D_GLIBCXX_USE_CXX11_ABI=0 -fno-sized-deallocation -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -fsyntax-only -Xclang -verify -ferror-limit=0 -Wno-invalid-noreturn -g -Xclang -load -Xclang /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O2 -fno-omit-frame-pointer -Werror -MD -MP -MF .deps/TestBadImplicitConversionCtor.o.pp /builds/worker/workspace/build/src/build/clang-plugin/tests/TestBadImplicitConversionCtor.cpp [task 2018-06-01T22:43:34.062Z] 22:43:34 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' [task 2018-06-01T22:43:34.062Z] 22:43:34 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' [task 2018-06-01T22:43:34.062Z] 22:43:34 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' [task 2018-06-01T22:43:34.062Z] 22:43:34 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/tests' /builds/worker/workspace/build/src/clang/bin/clang-5.0(_ZN5clang12ast_matchers11MatchFinder8matchASTERNS_10ASTContextE+0x2cd)[0x29865ad] /builds/worker/workspace/build/src/clang/bin/clang-5.0(_ZN5clang17MultiplexConsumer21HandleTranslationUnitERNS_10ASTContextE+0x2c)[0x1976e9c] /builds/worker/workspace/build/src/clang/bin/clang-5.0(_ZN5clang8ParseASTERNS_4SemaEbb+0x1c6)[0x2057036] /builds/worker/workspace/build/src/clang/bin/clang-5.0(_ZN5clang14FrontendAction7ExecuteEv+0x3f)[0x195215f] /builds/worker/workspace/build/src/clang/bin/clang-5.0(_ZN5clang16CompilerInstance13ExecuteActionERNS_14FrontendActionE+0x418)[0x1901ab8] /builds/worker/workspace/build/src/clang/bin/clang-5.0(_ZN5clang25ExecuteCompilerInvocationEPNS_16CompilerInstanceE+0x1086)[0x19cf0b6] /builds/worker/workspace/build/src/clang/bin/clang-5.0(_Z8cc1_mainN4llvm8ArrayRefIPKcEES2_Pv+0x3ff)[0x7fff6f] /builds/worker/workspace/build/src/clang/bin/clang-5.0(main+0x2b69)[0x7fe039] *** glibc detected *** /builds/worker/workspace/build/src/clang/bin/clang-5.0: free(): invalid pointer: 0x0000000003c8bf80 *** /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x7fb8f6918ead] ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x75bb6)[0x7f77e6ddabb6] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x6c)[0x7f77e6ddf95c] /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so(_ZN22KungFuDeathGripChecker5checkERKN5clang12ast_matchers11MatchFinder11MatchResultE+0x643)[0x7f77e6a78c03] /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x7fb1f1] ======= Memory map: ======== 00400000-03773000 r-xp 00000000 fc:00 1593850 /builds/worker/workspace/build/src/clang/bin/clang-5.0 03972000-03bea000 rw-p 03372000 fc:00 1593850 /builds/worker/workspace/build/src/clang/bin/clang-5.0 03bea000-03c9e000 rw-p 00000000 00:00 0 05971000-05a5b000 rw-p 00000000 00:00 0 [heap] 7fb8f0000000-7fb8f0021000 rw-p 00000000 00:00 0 7fb8f0021000-7fb8f4000000 ---p 00000000 00:00 0 7fb8f65c0000-7fb8f66de000 r-xp 00000000 fc:00 4234465 /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so 7fb8f66de000-7fb8f68de000 ---p 0011e000 fc:00 4234465 /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so 7fb8f68de000-7fb8f68e7000 rw-p 0011e000 fc:00 4234465 /builds/worker/workspace/build/src/obj-firefox/build/clang-plugin/libclang-plugin.so 7fb8f68e7000-7fb8f68fa000 rw-p 00000000 00:00 0 7fb8f68fa000-7fb8f6a7e000 r-xp 00000000 fc:00 2365198 /lib/x86_64-linux-gnu/libc-2.13.so 7fb8f6a7e000-7fb8f6c7d000 ---p 00184000 fc:00 2365198 /lib/x86_64-linux-gnu/libc-2.13.so 7fb8f6c7d000-7fb8f6c81000 r--p 00183000 fc:00 2365198 /lib/x86_64-linux-gnu/libc-2.13.so 7fb8f6c81000-7fb8f6c82000 rw-p 00187000 fc:00 2365198 /lib/x86_64-linux-gnu/libc-2.13.so 7fb8f6c82000-7fb8f6c87000 rw-p 00000000 00:00 0 7fb8f6c87000-7fb8f6c9d000 r-xp 00000000 fc:00 1722333 /builds/worker/workspace/build/src/clang/lib/libgcc_s.so.1 7fb8f6c9d000-7fb8f6e9c000 ---p 00016000 fc:00 1722333 /builds/worker/workspace/build/src/clang/lib/libgcc_s.so.1 7fb8f6e9c000-7fb8f6e9d000 rw-p 00015000 fc:00 1722333 /builds/worker/workspace/build/src/clang/lib/libgcc_s.so.1 7fb8f6e9d000-7fb8f6f1e000 r-xp 00000000 fc:00 2365217 /lib/x86_64-linux-gnu/libm-2.13.so 7fb8f6f1e000-7fb8f711d000 ---p 00081000 fc:00 2365217 /lib/x86_64-linux-gnu/libm-2.13.so 7fb8f711d000-7fb8f711e000 r--p 00080000 fc:00 2365217 /lib/x86_64-linux-gnu/libm-2.13.so 7fb8f711e000-7fb8f711f000 rw-p 00081000 fc:00 2365217 /lib/x86_64-linux-gnu/libm-2.13.so 7fb8f711f000-7fb8f7135000 r-xp 00000000 fc:00 2365270 /lib/x86_64-linux-gnu/libz.so.1.2.7 7fb8f7135000-7fb8f7334000 ---p 00016000 fc:00 2365270 /lib/x86_64-linux-gnu/libz.so.1.2.7 7fb8f7334000-7fb8f7335000 r--p 00015000 fc:00 2365270 /lib/x86_64-linux-gnu/libz.so.1.2.7 7fb8f7335000-7fb8f7336000 rw-p 00016000 fc:00 2365270 /lib/x86_64-linux-gnu/libz.so.1.2.7 7fb8f7336000-7fb8f7338000 r-xp 00000000 fc:00 2365206 /lib/x86_64-linux-gnu/libdl-2.13.so 7fb8f7338000-7fb8f7538000 ---p 00002000 fc:00 2365206 /lib/x86_64-linux-gnu/libdl-2.13.so 7fb8f7538000-7fb8f7539000 r--p 00002000 fc:00 2365206 /lib/x86_64-linux-gnu/libdl-2.13.so 7fb8f7539000-7fb8f753a000 rw-p 00003000 fc:00 2365206 /lib/x86_64-linux-gnu/libdl-2.13.so 7fb8f753a000-7fb8f7541000 r-xp 00000000 fc:00 2365251 /lib/x86_64-linux-gnu/librt-2.13.so 7fb8f7541000-7fb8f7740000 ---p 00007000 fc:00 2365251 /lib/x86_64-linux-gnu/librt-2.13.so 7fb8f7740000-7fb8f7741000 r--p 00006000 fc:00 2365251 /lib/x86_64-linux-gnu/librt-2.13.so 7fb8f7741000-7fb8f7742000 rw-p 00007000 fc:00 2365251 /lib/x86_64-linux-gnu/librt-2.13.so 7fb8f7742000-7fb8f7759000 r-xp 00000000 fc:00 2365245 /lib/x86_64-linux-gnu/libpthread-2.13.so 7fb8f7759000-7fb8f7958000 ---p 00017000 fc:00 2365245 /lib/x86_64-linux-gnu/libpthread-2.13.so 7fb8f7958000-7fb8f7959000 r--p 00016000 fc:00 2365245 /lib/x86_64-linux-gnu/libpthread-2.13.so 7fb8f7959000-7fb8f795a000 rw-p 00017000 fc:00 2365245 /lib/x86_64-linux-gnu/libpthread-2.13.so 7fb8f795a000-7fb8f795e000 rw-p 00000000 00:00 0 7fb8f795e000-7fb8f797e000 r-xp 00000000 fc:00 2365182 /lib/x86_64-linux-gnu/ld-2.13.so 7fb8f7b39000-7fb8f7b6f000 rw-p 00000000 00:00 0 7fb8f7b76000-7fb8f7b78000 rw-p 00000000 00:00 0 7fb8f7b78000-7fb8f7b7b000 r--p 00000000 00:00 0 [vvar] 7fb8f7b7b000-7fb8f7b7d000 r-xp 00000000 00:00 0 [vdso] 7fb8f7b7d000-7fb8f7b7e000 r--p 0001f000 fc:00 2365182 /lib/x86_64-linux-gnu/ld-2.13.so 7fb8f7b7e000-7fb8f7b7f000 rw-p 00020000 fc:00 2365182 /lib/x86_64-linux-gnu/ld-2.13.so 7fb8f7b7f000-7fb8f7b80000 rw-p 00000000 00:00 0 7ffdd6bd8000-7ffdd6bfa000 rw-p 00000000 00:00 0 [stack] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29a1bdf] /builds/worker/workspace/build/src/clang/bin/clang-5.0(_ZN5clang12ast_matchers8internal21BoundNodesTreeBuilder12visitMatchesEPNS2_7VisitorE+0x172)[0x29b76d2] /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29a1a8b] /builds/worker/workspace/build/src/clang/bin/clang-5.0[0x29b0b3b]

I'll just go with hiding the _M_start_thread symbols only. Let's figure out the more general std:: thing separately, if ever.

Pushed by mh@glandium.org: https://hg.mozilla.org/integration/autoland/rev/7c74241c18ea Don't export std::thread::_M_start_thread symbols with --enable-stdcxx-compat. r=froydnj

Hey Mike, what branches need uplift here?

Comment on attachment 8982390 [details] Bug 1464084 - Don't export std::thread::_M_start_thread symbols with --enable-stdcxx-compat. Approval Request Comment [Feature/Bug causing the regression]: bug 1389435 (switching the build to GCC 6), which means this has been a problem since 57. [User impact if declined]: possible infinite loop when creating /some/ kinds of threads, although it doesn't seem to be happening in the wild (according to crash-stats). [Is this code covered by automated tests?]: The test for bug 1454385 covers it (this is how this was found in the first place). [Has the fix been verified in Nightly?]: Yes, by reenabling the test for bug 1454385 [Needs manual test from QE? If yes, steps to reproduce]: N/A [List of other uplifts needed for the feature/fix]: N/A [Is the change risky?]: Not much [Why is the change risky/not risky?]: There is a change in how symbols are exported from libraries such as libmozavcodec.so, but that would tend to make things stricter than what they were before, forcing symbols being resolved from those libraries rather than system libraries that might have the same symbols, so overall, this should be a win. [String changes made/needed]: N/A

Comment on attachment 8982390 [details] Bug 1464084 - Don't export std::thread::_M_start_thread symbols with --enable-stdcxx-compat. Fixes a possible infinite loop caused by the switch to gcc6. Verified by the test on trunk which was previously failing due to this bug and is now successfully re-enabled in production. Approved for 61.0b12 and ESR 60.1.

Backed out for SM(pkg) libc-2.13.so crashes (seen on top of both Beta and ESR60). https://treeherder.mozilla.org/logviewer.html#?job_id=182006189&repo=mozilla-beta https://hg.mozilla.org/releases/mozilla-beta/rev/2e763d5ed8e5eba9cba54780a3142b9365b0cce3

The more I look at it, the more I'm puzzled. On the failing build, this is what happens per LD_DEBUG=all: 24392: symbol=malloc; lookup in file=./dist/bin/jsapi-tests [0] 24392: symbol=malloc; lookup in file=/lib/x86_64-linux-gnu/libpthread.so.0 [0] 24392: symbol=malloc; lookup in file=/builds/worker/workspace/sm-package/mozjs-60.0.2/obj-spider/dist/bin/libnspr4.so [0] 24392: symbol=malloc; lookup in file=/builds/worker/workspace/sm-package/mozjs-60.0.2/obj-spider/dist/bin/libplc4.so [0] 24392: symbol=malloc; lookup in file=/builds/worker/workspace/sm-package/mozjs-60.0.2/obj-spider/dist/bin/libplds4.so [0] 24392: symbol=malloc; lookup in file=/lib/x86_64-linux-gnu/libdl.so.2 [0] 24392: symbol=malloc; lookup in file=/lib/x86_64-linux-gnu/librt.so.1 [0] 24392: symbol=malloc; lookup in file=/lib/x86_64-linux-gnu/libz.so.1 [0] 24392: symbol=malloc; lookup in file=/lib/x86_64-linux-gnu/libm.so.6 [0] 24392: symbol=malloc; lookup in file=/usr/lib/x86_64-linux-gnu/libstdc++.so.6 [0] 24392: symbol=malloc; lookup in file=/lib/x86_64-linux-gnu/libgcc_s.so.1 [0] 24392: symbol=malloc; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0] 24392: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `malloc' [GLIBC_2.2.5] That is, mozjemalloc is not used. On a non failing build (with the backout): 2276: symbol=malloc; lookup in file=./dist/bin/jsapi-tests [0] 2276: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to ./dist/bin/jsapi-tests [0]: normal symbol `malloc' [GLIBC_2.2.5] so malloc is taken from jsapi-tests as expected, and is thus mozjemalloc. But here is the crazy thing: in both cases, malloc is exported from jsapi-tests: broken: $ objdump -T dist/bin/jsapi-tests | grep \ malloc$ 000000000051e400 g DF .text 0000000000000350 Base malloc non-broken: $ objdump -T dist/bin/jsapi-tests | grep \ malloc$ 000000000051e5a0 g DF .text 0000000000000350 Base malloc The LD_DEBUG=all output above are the first lines that differ between both cases. There is no difference between the logs before that. And if I download jsapi-tests from a mozilla-central sm-pkg build, malloc is correctly bound to jsapi-tests!

So whatever gold is doing to break this, it's because of gold being used for those builds. Which means there are two things to figure out: - why the sm-pkg builds on beta are using gold but the ones on central aren't? - what is the hidden difference between what gold and bfd ld produce that makes ld.so not find malloc in jsapi-tests?

> - why the sm-pkg builds on beta are using gold but the ones on central aren't? That's because bug 1455767 apparently broke defaulting to gold on local builds. Which brings to the problem that the sm-pkg builds shouldn't be using gold anyways.

So the problem is that gold doesn't do what would seem to be the right thing when it creates symbol versions (at least, it does things differently than both bfd ld and lld), *and* both objdump and readelf mask the problem by not displaying versions correctly in their output of the symbol table: they show the same thing as if there was no symbol version even when there is.

Filed: https://bugs.llvm.org/show_bug.cgi?id=37724 https://sourceware.org/bugzilla/show_bug.cgi?id=23267 https://sourceware.org/bugzilla/show_bug.cgi?id=23268

(In reply to Ryan VanderMeulen [:RyanVM] from comment #22) > Backed out for SM(pkg) libc-2.13.so crashes (seen on top of both Beta and > ESR60). > https://treeherder.mozilla.org/logviewer.html#?job_id=182006189&repo=mozilla- > beta > > https://hg.mozilla.org/releases/mozilla-beta/rev/ > 2e763d5ed8e5eba9cba54780a3142b9365b0cce3 These go away thanks to bug 1467041. Can this reland?