Missing symbols for GTK system libraries, libgtk-x11-* libgdk-x11-* and libglib-* Example crash: bp-013717d7-ebb1-4dac-a6a4-5b04f2100109 libgtk-x11-2.0.so.0.1200.0@0x224653 libgtk-x11-2.0.so.0.1200.0@0x224950 libgdk-x11-2.0.so.0.1200.0@0x1487a libglib-2.0.so.0.1400.1@0x2fd75 libglib-2.0.so.0.1400.1@0x2f605 libglib-2.0.so.0.1400.1@0x329a1 libglib-2.0.so.0.1400.1@0x32f5d nsAppShell::ProcessNextNativeEvent widget/src/gtk2/nsAppShell.cpp:147 ...

libgobject-* also

these aren't files we ship, and because each linux distro, version, minor, and user can have its own version of those files, there isn't a single source for these things. if we collected lsb_release -a, we could try to use that to retrieve files from vendors, but it'd require some database to maintain script rules or something....

Yeah, if anyone has a good idea about how to get decent symbols for the myriad of versions of libraries out there, I'm all ears. Note that Smokey has been doing this manually for OS X, where there are less versions to worry about. For Windows I have a script that pulls from the symbol server.

i just gave mats an outline of how to do it in theory for debian, it's derived from something i was already doing for debian. but we'll need the script you use to do uploads and it'll take him a bit of time (i'm sure) to get my script into working order. roughly the approach for debian is: setup a sources.list for a distro (or set of distros) setup an apt world update the world using the sources.list do a search for -dbg packages foreach package < get a list of all possible versions and add that package,version pair to a todo list > foreach of those {package,version}, < --download-only the thing foreach downloaded thing < dpkg-deb -x extract the package and upload the contents > > probably repeat this for each distro. note that this script works equally well or poorly for debian, ubuntu and maemo, and really any other debian derivative. conceptually, the same would apply to other package management systems, however you pretty much have to replace each line with a slightly different variant (sources.list => whatever, apt world => rpm world, ...).

(In reply to comment #2) "lsb_release -a" would be useful info to have in the crash data, regardless of if it helps with locating symbols or not.

Interestingly, Linux minidumps on trunk now contain the contents of /etc/lsb-release: http://mxr.mozilla.org/mozilla-central/source/toolkit/crashreporter/google-breakpad/src/client/linux/minidump_writer/minidump_writer.cc#387 but Socorro doesn't have any way to get that data out presently.

FWIW, Bruce Dawson at Valve has done some work on getting symbols for Linux system libraries: http://randomascii.wordpress.com/2013/03/05/symbols-on-linux-update-fedora-fixes/ It's possible we could leverage his work.

(In reply to Mats Palmgren (:mats) from comment #5) > (In reply to comment #2) > "lsb_release -a" would be useful info to have in the crash data, > regardless of if it helps with locating symbols or not. FYI, this is available in the "raw dump" tab since bug 602209 last year some time. For example: https://crash-stats.mozilla.com/report/index/da16047c-78fe-4d3b-8649-77ea12160610 has: "lsb_release": { "codename": "xenial", "description": "Ubuntu 16.04 LTS", "id": "Ubuntu", "release": "16.04" },

I did some work on scraping symbols out of Ubuntu's packages a few years ago, based on some work Bruce Dawson had done: https://github.com/luser/linux-symbol-scraping I'll try to find some time to revisit this in the near future now that I can schedule these things as Taskcluster hooks to run regularly. We should also be able to use Fedora's Darkserver now that we have Build IDs in our crash reports.

This may be a candidate feature for the tecken symbol server project.