The patch for bug 802400 made a substantial improvement to trobopan, reducing its metric from ~550000 to ~19000. I reviewed the dev-tree-management alerts between dev-tree-management Digest, Vol 47, Issue 143 ... dev-tree-management Digest, Vol 47, Issue 131 and could find no mention of the change. It seems to me that we occasionally miss alerts for both improvements and regressions...I do not know why. (The same patch produced modest improvements to tcheck2 and tcheck3, which were reported in dev-tree-management Digest, Vol 47, Issue 141).

this would be the magic graph server script

Here's a more subtle example: Android Ts regressed on m-i from about 3100 to about 4800 on November 30, 2012. The same regression is seen on m-c on December 1. I cannot find any dev-tree-management reports for these regressions. However, what appears to be the same regression on mozilla-aurora on January 8 was reported in dev-tree-management Digest, Vol 49, Issue 49: Message: 4 Date: Wed, 09 Jan 2013 13:41:20 -0000 From: nobody@cruncher.build.mozilla.org To: dev-tree-management@lists.mozilla.org Subject: <Regression> Mozilla-Aurora - Ts - Android 2.2 (Native) - 54.3% Message-ID: <20130109134120.532631042ED@cruncher.srv.releng.scl3.mozilla.com> Content-Type: text/plain; charset="us-ascii" Regression: Mozilla-Aurora - Ts - Android 2.2 (Native) - 54.3% increase ----------------------------------------------------------------------- Previous: avg 3145.322 stddev 356.677 of 30 runs up to revision 1903a40f342d New : avg 4852.908 stddev 298.683 of 5 runs since revision a459eaaea5ae Change : +1707.586 (54.3% / z=4.787) Graph : http://mzl.la/13gF1KU Changeset range: http://hg.mozilla.org/releases/mozilla-aurora/pushloghtml?fromchange=1903a40f342d&tochange=a459eaaea5ae I say "more subtle example" because in this case there is a lot of noise: Ts went from a range of about 3000-3200 to a range of about 3000-5500.

we miss alerts on graph server for a variety of reasons, some are bugs in graph server, some are suppression code in the algorithm which seems to cause us to miss other alerts on other platforms as well. We are moving to autophone and the regression detection will be much different.