Closed Bug 1520219 Opened 6 years ago Closed 4 years ago

Intermittents case study: Bug 1519038

Categories

(Testing :: General, defect, P3)

Version 3
defect

Tracking

(Not tracked)

RESOLVED INACTIVE

People

(Reporter: gbrown, Assigned: gbrown)

References

Details

Bug 1519038 records over 100 failures between January 9 and January 15.

Additional failures of the same type were recorded in:

bug 1337253
bug 1430083
bug 1449582

Why did we use so many bugs?

This was a very frequent failure in a test that was recently modified and which failed TV; why didn't the sheriffs notice / why wasn't bug 1498195 backed out?

Should the IF bot have marked it disabled-recommended? needswork?

Why did attempts to find a regression range fail? (And did those delay or confuse efforts to resolve the failures?) (Note that the failing test moved from one chunk to another, so https://bugzilla.mozilla.org/show_bug.cgi?id=1519038#c6 doesn't capture the issue.)

Let's use this as a case study to see if the IF bot rules can be improved and whether policy changes would improve outcomes.

Assignee: nobody → whole.grains
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → INACTIVE
You need to log in before you can comment on or make changes to this bug.