Closed Bug 1110285 Opened 10 years ago Closed 10 years ago

titanic could use a intermittent mode to help narrow down intermittents

Categories

(Testing :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jmaher, Unassigned, Mentored)

References

Details

when tests start failing intermittently, it is usually related to something unrelated which changed in the product. titanic can help us figure this out, specifically by retriggering the specific job 40 times on a series of pushes. I see this as the steps required: 1) user inputs revision, branch, buildername 2) titanic generates list of 19 revisions prior (+ given one) 3) titanic then schedules 39 retriggers for given buildername on each of the revisions from step 2. 4) titanic then stores a link to treeherder with fromchange= and tochange= to show the range of tests for manual verification Design considerations - We will want to eventually integrate a smarter system here such that we can bisect - this would entail doing the retriggers on a select subset of revisions, then determining which is the smaller range and repeating the process until we have the revision that the intermittent started at. Cautions - * we could find the intermittent starting from a merge- in this case we will need to run this again on the branch from which the merge came from. I can see this as turning into a fun mess, but using smaller ranges and logic to determine which revisions to retrigger on could save us trouble. * if we don't have a build, titanic will automatically build. I am not sure if we want to schedule this automatically, or just report that there is no build available and retrigger where we have preexisting tests.
as we have been renewing our interest in scripted launching of jobs, a lot of little pieces have been put in place and this has been resolved in the mozilla_ci_tools repository: https://github.com/armenzg/mozilla_ci_tools/blob/master/scripts/trigger_range.py
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Whiteboard: [good first project][lang=python]
You need to log in before you can comment on or make changes to this bug.