Closed
Bug 1292338
Opened 8 years ago
Closed 8 years ago
Reftest failures of the form "testfilename.html == referencefilename.html" should search for bugs with testfilename.html
Categories
(Tree Management :: Treeherder, defect)
Tree Management
Treeherder
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: philor, Assigned: jgraham)
References
Details
Attachments
(1 file)
Finally realized why webgl reftest failures are now all getting "too many bug suggestions": we switched reftest logging to list both the test filename and the reference filename, like "REFTEST PROCESS-CRASH | http://10.0.2.2:8854/tests/dom/canvas/test/reftest/webgl-resize-test.html == http://10.0.2.2:8854/tests/dom/canvas/test/reftest/wrapper.html?green.png | application crashed [@ jemalloc_crash]"", and so when treeherder looks for the test filename, it takes the thing after the rightmost / between the first and second |, which is wrapper.html?green.png, which is the reference file for tens or hundreds of different webgl reftests.
The "everything should be fixed upstream" view of that would be that the ouput should instead be "REFTEST BADTHING | tiresomepath/testfilename | == referencefilename bad thing that happened", that the test is testfilename and the == referencefilename bit is the equivalent of an individual is(this, that) in a mochitest.
Or, treeherder could just hack up the hacky filename finder to know that for reftests it's not the rightmost, it's the rightmost to the left of the first ==/!=.
Either way, this is a huge regression, and the reason why I've been filing five or six or ten duplicates of the same webgl crash lately.
Comment 1•8 years ago
|
||
Assignee | ||
Updated•8 years ago
|
Attachment #8777969 -
Flags: review?(emorley)
Comment 2•8 years ago
|
||
Comment on attachment 8777969 [details]
[treeherder] mozilla:error_summary_reftest > mozilla:master
This is a deviance from the (albeity crufty) log format we've used for years.
I'm not really sure we should be working around this in Treeherder.
If people want to propose a new error message format for unstructured logs, they are welcome to, but let's have proper mailing list discussion, choice of new format and support added to all places where error lines are parsed first.
Attachment #8777969 -
Flags: review?(emorley)
Assignee | ||
Comment 3•8 years ago
|
||
Comment on attachment 8777969 [details]
[treeherder] mozilla:error_summary_reftest > mozilla:master
Per the conversation on irc it sounds like you might be OK with this after all.
Attachment #8777969 -
Flags: review?(emorley)
Updated•8 years ago
|
Attachment #8777969 -
Flags: review?(emorley) → review+
Updated•8 years ago
|
Assignee: nobody → james
Comment 4•8 years ago
|
||
Commit pushed to master at https://github.com/mozilla/treeherder
https://github.com/mozilla/treeherder/commit/be1ca5872bb56ba78fb92b7a03fb3bfccd6a5ca7
Bug 1292338 - Remove reftest references from error summary search terms. (#1761)
Updated•8 years ago
|
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•