I want to do a more detailed analysis of the crash reports we're receiving which come from the hang detector. For now, let's just stick with Windows reports, because it's easier to identify them. Query version is 3.6.3plugin1 or 3.6.4pre or 3.6.4 Crash reason is EXCEPTION_BREAKPOINT Past week. Even if it's just a set of crash IDs, I can run correlations using the JSON and raw dump data to try and break things down better. I'd love a more detailed report, but in order for it to be truly useful I'll need the detailed crash stack so I can try and construct some better signature data.

Oh, and if it can be automated to send me this report every few days for a while, this would be really useful.

one thought would be to add crash_reason to the .csv files. that would allow a fair bit of analysis on its own and the .csv files also have uuid_url for digging up the individual reports and JSON raw dumps. current contents of the .csv files are and we could tack crash_reason on the end. 1 signature 2 url 3 uuid_url 4 client_crash_date 5 date_processed 6 last_crash 7 product 8 version 9 build 10 branch 11 os_name 12 os_version 13 cpu_name 14 address 15 bug_list 16 user_comments 17 uptime_seconds 18 email 19 adu_count 20 topmost_filenames 21 addons_checked 22 flash_version sanitized (urls edited) reports get published every night under http://people.mozilla.com/crash_analysis/ with dbaron's correlation reports. a simple awk, grep, or python script could extract all the "EXCEPTION_BREAKPOINT" entries along with uuid_url or any other fields of interest. Would this option work?

Is there data on the CSV files? Are they a sample of the crashes, or the full set? I don't really know anything about them, but since I'm going to have to pull full stack data and do manual HangID correlation until socorro grows those features, I was looking for the simplest solution.

(In reply to comment #3) > Is there data on the CSV files? Its just a summary of each crash report with the 22 items in comment 2. > Are they a sample of the crashes, or the full > set? full set of all processed crashes each day. (midnight to midnight) > I don't really know anything about them, but since I'm going to have to > pull full stack data and do manual HangID correlation until socorro grows those > features, I was looking for the simplest solution. it would be a pretty simple 4 or 5 line script to grab just the data you want, when you want it, from this source. scp or wget the_file from people gunzip the_file | awk -F\t '$23 ~ /EXCEPTION_BREAKPOINT/ {printf "%s\t%s\n" $1,$3}' would give you a tab separated list of all signatures uuid_urls for all the hang reports. If you decide you want to add os version or product version to help in filtering out the reports that's under your control, and not another request to the socorro team. We are using these .csv files for other report prototyping like jst's new crashes stuff http://people.mozilla.com/~jst/new-crashes/Firefox/latest/ and a few other one-off and experimental analysis reports.

Sure, but the "data" I really wanted was "how can I get them"?

sounds like HangID might be another good addtion to the .csv files if we can get that.

FWIW, I'm now slurping up the data I need from the CSV files combined with the .json .jsonz files and it's in a couch. I don't need anything else for this bug, although chofmann may still want the HangID/crash reason stuff in the CSV if that isn't too hard to do.

yeah, I'm still interested in crash reason being added to the .csv

1.6.2 backend changes are getting pushed to staging. The included changes are: 1 - processor stores HangID in the reports table - hangid is indexed 2 - dailyUrl report now includes 'HangID' and 'reason' Daily URL format: 1 signature 2 url 3 uuid_url 4 client_crash_date 5 date_processed 6 last_crash 7 product 8 version 9 build 10 branch 11 os_name 12 os_version 13 cpu_name 14 address 15 bug_list 16 user_comments 17 uptime_seconds 18 email 19 adu_count 20 topmost_filenames 21 addons_checked 22 flash_version 23 hangid 23 reason

cool! thanks lars.

marking this as resolved so we know we are "go" for production push.

did 1.6.2 get pushed to production last night? I'm not seeing hangid and reason in the .csv's produced early this morning.

bah, this was completed in code, but inadvertently omitted from the explicit production push instructions. I'm refiling a bug to get that done. See Bug 562856