Use minidump-stackwalk's JSON output for mozcrash.py
Categories
(Toolkit :: Crash Reporting, enhancement)
Tracking
()
People
(Reporter: ted, Unassigned)
References
(Blocks 1 open bug)
Details
Attachments
(1 file)
(deleted),
text/x-phabricator-request
|
Details |
Updated•5 years ago
|
Comment 1•3 years ago
|
||
We're replacing minidump_stackwalk
with the oxidized version soon. After we've done that we could switch the output to JSON and we should be able to easily implement what Ted suggested in comment 0.
Reporter | ||
Comment 2•3 years ago
|
||
🙌 I'm obviously not doing any of the work so you don't have to listen to me, but if you do that it might be useful to tweak the Rust minidump-stackwalk
so you can ask it to spit out both the human-readable and JSON output, maybe to separate files or something (IDK, whatever) so that you could have mozcrash
parse the JSON to generate a signature (ideally just reusing the existing siggen
module) and also put the human-readable output somewhere in the CI logs for humans to read, without having to write code to replicate that output from the JSON format.
Comment 3•3 years ago
|
||
Yes last night i was already considering adding a --cyborg=path/to/write/json option
Updated•3 years ago
|
Updated•3 years ago
|
Comment 4•3 years ago
|
||
Repurposing this bug:
It sucks that mozcrash.py parses minidump-stackwalk's human readable output instead of just requesting the machine-readable JSON (which also contains way more details!). It causes problems.
The only thing mildly blocking making this change is that the human-readable output is genuinely nice to have for humans. There are 4 options to resolve this:
- Ideally: implement cyborg output: https://github.com/luser/rust-minidump/issues/296
- Lazily: just run minidump-stackwalk twice, once for each mode (wasteful, but at least the disk cache reduces a lot of the waste)
- Rudely: pass the --pretty flag so the JSON is vaguely human-readable and just dump that in the logs
- Chaotically: have mozcrash.py synthesize its own human-friendly format from the json (bad allocation of human effort, but viable)
Updated•3 years ago
|
Comment 5•3 years ago
|
||
Having Socorro-compatible JSON here paves the way to finally fix bug 867571 the importance of which cannot be underestimated: fixing it would finally allow us to automatically tie the crashes we see in automation with those we see in the wild.
Comment 6•3 years ago
|
||
Implemented --cyborg output in https://github.com/luser/rust-minidump/pull/316
Now just need to implement the mozcrash.py infra (and update the toolchain)
Comment 7•3 years ago
|
||
Note: the schema for the json output is """documented""" here: https://github.com/luser/rust-minidump/blob/a224c912df9ddde3a956f9796176b3aacde69b1a/minidump-processor/src/process_state.rs#L590
Everything we need should be in the top-level crashing_thread field.
Updated•3 years ago
|
Comment 8•3 years ago
|
||
This gets us the crash report in a stable machine-readable format which is much
more reliable to parse and analyze.
Also bumps the version of minidump-stackwalk to get the cyborg feature and
some CLI fixes.
This moves signature generation as a responsibility from rust-minidump
to mozcrash.py. In the long-term this is desirable, as it would allow us to
keep the implementation in sync with socorro. But in the short-term the output
is worse until we implement that functionality.
Comment 9•3 years ago
|
||
The above patch is semi-WIP. It works, but the signature generation is garbage. This is because the JSON output doesn't give you a signature, it just gives you the parts to synthesize it yourself.
We need to do one of the following to make it decent:
- Hand-roll signature generation in mozcrash.py (yuck).
- Make minidump-stackwalk include a synthesized signature to the json output (kinda weird, but the machinery is already there for human output. It just wouldn't be in sync with socorro, maybe the best cost+benefit in the short term?).
- Use https://github.com/willkg/socorro-siggen to generate signatures (Will has concerns about maintenance/vendoring).
- Implement Bug 828452 (add a signature generation web api to socorro) (arguably ideal, since the end-goal is to be in sync with socorro, but a lot of upfront work).
Comment 10•3 years ago
|
||
Just before the end of the year, I threw together a half-baked signature generation web api for Socorro. I need to finish and document it. I think I'll get to it in 2022h1. Maybe June.
Is this bug really impactful such that I should shove some things around in my queue and get to it earlier?
Comment 11•3 years ago
|
||
No, we lived without it for ages so it's not urgent.
Updated•2 years ago
|
Comment 12•2 years ago
|
||
The bug assignee is inactive on Bugzilla, so the assignee is being reset.
Description
•