The scope of this bug is: * A intermittent detector that, given multiple retriggers of the same job of which at least one is green, will mark test failures seen in the other jobs as possible targets for automatic classification. * An automatic classifier that will take individual test failures and compare them to known failures (typically from the detector above) and mark matches where the test and error message are identical.

Commit pushed to master at https://github.com/mozilla/treeherder https://github.com/mozilla/treeherder/commit/7b6fa254025a5b0688b6c6c7cbc4c700adb6169f Bug 1204942 - First cut at autoclassify / intermittent orange detection. This adds an autoclassify command and a detect_intermittents command. The former is designed to take an incoming job with an error summary and look for existing results marked as intermittent that are a close match for the new result. At present only one matcher is implemented; this requires an exact match in terms of test name, result and error message. Matching is also constrained to be based on single lines; it is anticipated that future iterations may add support for matching on groups of lines. The detect_intermittents command is designed to take a group of jobs running on the same push and with the same build job (i.e. same testsuite, same chunk, etc.) and look for new intermittents to add to the database. This currently only looks for test failures where there is at least one green job and one non-green job. There is currently no UI for seeing matches or for adding new prototypical intermittents as match candidates. There is also no integration with bugzilla; future development should add association of frequent intermittents with bugs.