Intermittents case study: Bug 1519038
Categories
(Testing :: General, defect, P3)
Tracking
(Not tracked)
People
(Reporter: gbrown, Assigned: gbrown)
References
Details
Bug 1519038 records over 100 failures between January 9 and January 15.
Additional failures of the same type were recorded in:
bug 1337253
bug 1430083
bug 1449582
Why did we use so many bugs?
This was a very frequent failure in a test that was recently modified and which failed TV; why didn't the sheriffs notice / why wasn't bug 1498195 backed out?
Should the IF bot have marked it disabled-recommended? needswork?
Why did attempts to find a regression range fail? (And did those delay or confuse efforts to resolve the failures?) (Note that the failing test moved from one chunk to another, so https://bugzilla.mozilla.org/show_bug.cgi?id=1519038#c6 doesn't capture the issue.)
Let's use this as a case study to see if the IF bot rules can be improved and whether policy changes would improve outcomes.
Assignee | ||
Updated•4 years ago
|
Description
•