Query triage owners for per-component test efficiencies
Categories
(Testing :: General, enhancement, P3)
Tracking
(Not tracked)
People
(Reporter: gbrown, Unassigned)
References
(Depends on 1 open bug)
Details
Sometimes features are rewritten, replaced, or become obsolete but we forget to turn off the related tests. Sometimes new tests are written that replace the need for earlier tests, but we keep running both versions. It is difficult to find cases like this by any automated method, but domain experts may be aware of inefficiencies within their components.
I would like to use the source-file -> bugzilla-component -> triage-owner relationship to contact the triage owners associated with tests and
- remind them of the number of tests associated with their bugzilla components;
- ask them to reflect on whether we are getting good value from those tests;
- ask them to identify any tests that can be discontinued;
- ask them to identify any tests that could be scheduled less often.
Reporter | ||
Comment 1•6 years ago
|
||
Ideas for contacting triage owners:
- direct email
- create a bug, needinfo triage-owner
- survey invite (surveymonkey)
Reporter | ||
Comment 2•6 years ago
|
||
Triage owners are busy; I want to be sensitive to that, so:
- brief communication
- make it clear that we're not asking for a comprehensive review
- we are happy to write patches or anything else to help
Reporter | ||
Updated•6 years ago
|
Reporter | ||
Comment 3•6 years ago
|
||
Other "causes" could be piggy-backed on this sort of communication, such as:
- Intermittents: "In <time period>, the worst offending intermittent test failures in your component are <bugs/tests/# failures>. Let us know if you would like any of these tests disabled."
Comment 4•6 years ago
|
||
Vaguely related, once we fix bug 1421972 (and similar for other test harnesses), it would be interesting to automatically find which tests are the slowest for each component, and suggest that triage owners have a look at a profile of these specific tests, to ensure that they are actually behaving as expected.
Reporter | ||
Updated•6 years ago
|
Updated•6 years ago
|
Updated•6 years ago
|
Reporter | ||
Updated•5 years ago
|
Updated•2 years ago
|
Description
•