Open Bug 1756895 Opened 3 years ago Updated 1 year ago

[tracking] Simplify usage of bugs to track intermittent test failures

Categories

(Testing :: General, enhancement)

Default
enhancement

Tracking

(Not tracked)

People

(Reporter: jmaher, Unassigned)

References

(Depends on 14 open bugs)

Details

(Keywords: meta)

This bug will be for tracking work to revisit our methods for tracking and analyzing intermittent failures.

The current model is each unique failure will be matched to a bug and in general we want to switch to each unique test case will have a single bug.

This will start with xpcshell and ignore failures that are not test cases (leaks, crashes, etc.)

Depends on: 1756897
Depends on: 1756898
Depends on: 1757222
Depends on: 1757224
Depends on: 1757225
Depends on: 1757660
Depends on: 1757663
Depends on: 1757959
Depends on: 1758984
Depends on: 1763489

next steps on this bug is to analyze the effectiveness of these bugs for developers fixing failures vs not. That was going to wait 30 days- I think that might have to wait longer. I will analyze this next week (After the 10th).

If all is looking good, we can expand to other harnesses- mochitest browser-chrome and devtools seem high on my list. Will evaluate and propose some next steps.

Depends on: 1821515
Depends on: 1821516
Depends on: 1821520
Depends on: 1821522
Depends on: 1821525
Depends on: 1821527
Depends on: 1821528
Depends on: 1821531
Depends on: 1821533
Depends on: 1821536
Depends on: 1850769
Depends on: 1850779
Depends on: 1850785
You need to log in before you can comment on or make changes to this bug.