Open Bug 1519171 Opened 6 years ago Updated 2 years ago

Query triage owners for per-component test efficiencies

Categories

(Testing :: General, enhancement, P3)

Version 3
enhancement

Tracking

(Not tracked)

People

(Reporter: gbrown, Unassigned)

References

(Depends on 1 open bug)

Details

Sometimes features are rewritten, replaced, or become obsolete but we forget to turn off the related tests. Sometimes new tests are written that replace the need for earlier tests, but we keep running both versions. It is difficult to find cases like this by any automated method, but domain experts may be aware of inefficiencies within their components.

I would like to use the source-file -> bugzilla-component -> triage-owner relationship to contact the triage owners associated with tests and

  • remind them of the number of tests associated with their bugzilla components;
  • ask them to reflect on whether we are getting good value from those tests;
  • ask them to identify any tests that can be discontinued;
  • ask them to identify any tests that could be scheduled less often.

Ideas for contacting triage owners:

  • direct email
  • create a bug, needinfo triage-owner
  • survey invite (surveymonkey)

Triage owners are busy; I want to be sensitive to that, so:

  • brief communication
  • make it clear that we're not asking for a comprehensive review
  • we are happy to write patches or anything else to help
Priority: -- → P3

Other "causes" could be piggy-backed on this sort of communication, such as:

  • Intermittents: "In <time period>, the worst offending intermittent test failures in your component are <bugs/tests/# failures>. Let us know if you would like any of these tests disabled."

Vaguely related, once we fix bug 1421972 (and similar for other test harnesses), it would be interesting to automatically find which tests are the slowest for each component, and suggest that triage owners have a look at a profile of these specific tests, to ensure that they are actually behaving as expected.

Depends on: 1577302
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.