The fix for bug 539516 doesn't help on Tinderbox like it does on my machine. The functions are still approximate and line numbers still aren't shown :( Mac, old (fix-macosx-stack.pl): http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266785258.1266788966.13481.gz&fulltext=1 Mac, new (fix_macosx_stack.py): http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266786975.1266789121.13863.gz&fulltext=1 I suspect this is due to the packaging that separates out symbols for breakpad. Maybe it should leave the symbols in for debug builds? Compare assertion failures from these two Linux runs on the same build: Linux, Bd (before packaging): http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266786447.1266788049.11030.gz&fulltext=1 Linux, Rd (after packaging): http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266787686.1266792281.21931.gz&fulltext=1

The test harnesses using automation.py take an optional argument to specify the symbol location. Could fix_macosx_stack.py do the same ?

Probably, by filling in the "separate_debug_file_for" stub. But I have no idea how to do that.

Ted, sounds like this should go to Testing::<Something> ?

Apparently |atos| can't handle dSYM, and you have to use |dwarfdump| or |gdb| instead if all you have is dSYM files. Argh. http://lists.apple.com/archives/xcode-users/2008/Feb/msg00525.html http://wincent.com/a/products/atosym/ http://boredzo.org/symbolicator/

Lame. I guess we could not strip the debug builds, but that'd probably wind up with huge binaries.

(In reply to comment #4) > Apparently |atos| can't handle dSYM, and you have to use |dwarfdump| or |gdb| > instead if all you have is dSYM files. Argh. You pointed atos at the dSYM file and not the stripped library, right?

I think I did. I tested on 10.5 with a build from Tinderbox along with its tests and symbols files.

Mac Bd with an assertion failure: http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266874895.1266876347.28131.gz&fulltext=1 Mac Rd with assertion failures: http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1266872979.1266876036.27040.gz&fulltext=1

Even though dSYM is now the standard format for debugging symbols on Mac, not all debugging tools know how to use them, and very few tools know how to find them :( I expect Valgrind to have similar issues.

Of course, it would be nice for Valgrind to work with opt builds too, and we're definitely going to keep stripping those.

Actually, ironically, we stopped stripping our nightly/hourly opt builds on Mac, so they work better with Shark. (Release builds are still stripped.)

Split out bug 547873 for Linux.

Since we're already using --disable-install-strip for the OS X Nightly builds, we should probably just try setting that for debug builds and seeing what it does. IIRC, it only increased the size of the package by about 5MB for opt builds. We could be even smarter, and stop shipping the stripped out debug bits in the symbols package if we're not stripping the package to begin with (to save space), but that's not necessary.

We should probably switch from atos to something based on breakpad or gdb. That could take care of Linux too. Valgrind can take care of itself: https://bugs.kde.org/show_bug.cgi?id=200214.

The Breakpad symbol file format is pretty trivial: http://code.google.com/p/google-breakpad/wiki/SymbolFiles I have a Python script laying around here somewhere that can turn module+offset into a function name given a symbol file.

Ted, it would be great if you could dig up that Python script. I can help turn it into a replacement for fix_macosx_stack.py.

Here it is. It only handles parsing a single symbol file right now, and looking up functions by address within that file. It expects to take a symbol file as its only commandline argument, and hex addresses on stdin. It will look up each address from stdin and print the function name if it finds it. (I think it also accepts lines as if they came from minidump_stackwalk with missing symbols, which is why there's a second regex there.) The only tricky part to making this work with a symbol directory like we produce as part of the build would be that Breakpad looks up symbols by module ID, which on OS X is a hash of some bits of the file, IIRC. You can probably cheat, though, since there should only be one symbol file per module, and the symbol store looks like <filename>/<UUID>/<filename>.sym, and there should only be one UUID there.

Awesome. I added support for line numbers, mashed it up with fix_macosx_stack.py, and uploaded it to bug 570287. Thanks, Ted!

Can we WONTFIX this given the status of bug 570287?

Yes.