FX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS (Average frame interval during tab open animation of about:newtab (preview=on), when other tabs are unaffected) seem to have regressed over the last nightly versions. By comparing nightly32 and nightly33 on http://telemetry.mozilla.org/ against previous nightly versions one can notice by looking at the 25th, 50th and 75th percentiles that the distributions of nightly32 & nightly33 seem to be slightly shifted to the right. A quick look at the Telemetry data seems to point to a slightly stronger regression for Intel GPUs followed by ATI’s and finally Nvidia’s chips.

(In reply to Roberto Agostino Vitillo (:rvitillo) from comment #0) > By comparing nightly32 and nightly33 on http://telemetry.mozilla.org/ Vladan, Mark, It would be useful to be able to get a direct link to said comparison... where do we file this bug or who do we talk to? It would _really_ help narrowing it down to a specific date or few dates. On this case, however, I could think of at least 2 possible issues for this: OMTC and the new tab's search bar. Matt, was OMTC turned on by default on Firefox 33? If it's OMTC, then it's a known issue and roughly being worked on (straitening out the kinds of OMTC). Specifically bug 1024643 and bug 940845 might have helped mitigating the tab animation issue, as well as a new Intel GPU driver - which only affects the last 3 generations and which was released about 3 weeks ago. If it's not OMTC, it could be Drew's newta'b search bar at bug 962490, and the regression is tracked at bug 1004429. Already few improvements have landed, the most notable is probably bug 1019990 (better preload "system"). Drew, Do you have a rough date when you think most of the regression from the search bar was gone? But In general I'm not sure what we can do with such regression reports. It's very hard to catch a regression between firefox versions because there are so much changes in such time period, so other than acknowledging that we've regressed, usually I think there's not much we can do.

(In reply to Avi Halachmi (:avih) from comment #1) > (In reply to Roberto Agostino Vitillo (:rvitillo) from comment #0) > > By comparing nightly32 and nightly33 on http://telemetry.mozilla.org/ > > Vladan, Mark, > It would be useful to be able to get a direct link to said comparison... Avi, here are some links for a comparison between e.g. nightly29 and nightly33: Nightly29: http://telemetry.mozilla.org/#filter=nightly%2F29%2FFX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS%2Fsaved_session%2FFirefox%2FWINNT&aggregates=5th%20percentile!25th%20percentile!median!75th%20percentile&evoOver=Builds&locked=true&sanitize=true&renderhistogram=Graph Nightly33: http://telemetry.mozilla.org/#filter=nightly%2F33%2FFX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS%2Fsaved_session%2FFirefox%2FWINNT&aggregates=5th%20percentile!25th%20percentile!median!75th%20percentile&evoOver=Builds&locked=true&sanitize=true&renderhistogram=Graph Nightly29 vs Nightly33: http://telemetry.mozilla.org/#filter=nightly%2F29%2FFX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS%2Fsaved_session%2FFirefox%2FWINNT!nightly%2F33%2FFX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS%2Fsaved_session%2FFirefox%2FWINNT&aggregates=median&evoOver=Builds&locked=true&sanitize=true&renderhistogram=Graph

(In reply to Roberto Agostino Vitillo (:rvitillo) from comment #2) > Nightly29 vs Nightly33: > http://telemetry.mozilla.org/ > #filter=nightly%2F29%2FFX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS%2Fsaved_ses > sion%2FFirefox%2FWINNT! > nightly%2F33%2FFX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS%2Fsaved_session%2FF > irefox%2FWINNT&aggregates=median&evoOver=Builds&locked=true&sanitize=true&ren > derhistogram=Graph To be honest, either I don't know what to look at with these graphs, or I don't think I see a meaningful difference at this link ( http://mzl.la/1qqXWQx ). The values at both graphs are about 18 (ms) most of the time. Your initial post was about firefox 32 vs 33, but now your links are about 29 vs 33. Also, I'm unable to see any histograms or comparisons of such - where you claim most of the difference lies. I'm not saying that your report is incorrect, just that I don't know how to see what you see, or how to conclude what you conclude.

(In reply to Avi Halachmi (:avih) from comment #3) > (In reply to Roberto Agostino Vitillo (:rvitillo) from comment #2) > > Nightly29 vs Nightly33: > > http://telemetry.mozilla.org/ > > #filter=nightly%2F29%2FFX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS%2Fsaved_ses > > sion%2FFirefox%2FWINNT! > > nightly%2F33%2FFX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS%2Fsaved_session%2FF > > irefox%2FWINNT&aggregates=median&evoOver=Builds&locked=true&sanitize=true&ren > > derhistogram=Graph > > To be honest, either I don't know what to look at with these graphs, or I > don't think I see a meaningful difference at this link ( > http://mzl.la/1qqXWQx ). The values at both graphs are about 18 (ms) most of > the time. > > Your initial post was about firefox 32 vs 33, but now your links are about > 29 vs 33. > > Also, I'm unable to see any histograms or comparisons of such - where you > claim most of the difference lies. > > I'm not saying that your report is incorrect, just that I don't know how to > see what you see, or how to conclude what you conclude. Fair enough, the last view might be misleading. Ideally what we would want to do is to compare the distributions by normalizing and displaying them one over the other, but that's not possible with the current dashboard. My initial post was about 32 and 33 vs previous nightly versions. Both 32 and 33 exhibit a shift in their distribution when comparing them to distributions belonging to earlier nightly versions (<= 31). Now, for the sake of argument, let's compare nightly33 vs nightly29. Since we can't overlay the histograms, let's just look at them individually. nightly33: http://telemetry.mozilla.org/#filter=nightly%2F33%2FFX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS%2Fsaved_session%2FFirefox%2FWINNT&aggregates=5th%20percentile!25th%20percentile!median!75th%20percentile&evoOver=Builds&locked=true&sanitize=true&renderhistogram=Graph nightly29: http://telemetry.mozilla.org/#filter=nightly%2F29%2FFX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS%2Fsaved_session%2FFirefox%2FWINNT&aggregates=5th%20percentile!25th%20percentile!median!75th%20percentile&evoOver=Builds&locked=true&sanitize=true&renderhistogram=Graph If you now look at the summaries of the histograms you will see the following percentiles: nightly 29: nightly33 5th Percentile 15 16 25th Percentile 16 18.01 Median 18.07 20.59 75th Percentile 24.49 31.09 95th Percentile 67.66 81.78 As you can see the percentiles in nightly33 are shifted to the right wrt. nightly29, does that make sense?

Also, by looking at the timeline of the median of FX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS in nightly32, you can notice that the regression seems to have appeared around the 21st of May.

(In reply to Roberto Agostino Vitillo (:rvitillo) from comment #4) > If you now look at the summaries of the histograms you will see the > following percentiles: > > nightly 29: nightly33 > ... > > As you can see the percentiles in nightly33 are shifted to the right wrt. > nightly29, does that make sense? I can see the shift, seems like a fact, so sense should have nothing to do with it ;) However, the 29 link covers few months, while the 33 link covers 2 _weeks_. Do we treat here a version as a monolithic thing (i.e. the performance of a single build)? or do we also track a version's performance over time - which could change throughout the lifetime of a version, especially while it's at Nightly or Aurora? In general, I think that using a release channel together with a version number (e.g. "Nightly" with "Firefox 29") is a major source for confusion and inaccuracies in communications. A release channel keeps its name, but the version which it contains changes every cycle. IMO it would be _much_ better to use version number only and just ignore the channel completely (unless as a complementary - e.g. "when Firefox 25 was at Aurora..."). To summarize: - Let's use version numbers only. - To make it clearer, note if you're observing a specific build, or processing the entire lifetime of a version - which could change over time and therefore represent different performance values into one histogram. - I'm still not sure if you point to a regression between a specific build of firefox 33 or 32, or the entire lifetime of 33 or 32, and to what. - Even if we can see this regression and agree on it, it's still not clear to me what we can do about it. Tracking regression when we know the exact changeset is hard enough work, so being able to do something about a regression between (possibly few) releases really seems like it's for the record only. (In reply to Roberto Agostino Vitillo (:rvitillo) from comment #5) > Also, by looking at the timeline of the median of > FX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS in nightly32, you can notice that > the regression seems to have appeared around the 21st of May. OK, this is finally workable, and it's clearly OMTC which landed around May 20th. It's being worked on.

(In reply to Avi Halachmi (:avih) from comment #6) > - Even if we can see this regression and agree on it, it's still not clear > to me what we can do about it. Tracking regression when we know the exact > changeset is hard enough work, so being able to do something about a > regression between (possibly few) releases really seems like it's for the > record only. > > (In reply to Roberto Agostino Vitillo (:rvitillo) from comment #5) > > Also, by looking at the timeline of the median of > > FX_TAB_ANIM_OPEN_PREVIEW_FRAME_INTERVAL_MS in nightly32, you can notice that > > the regression seems to have appeared around the 21st of May. > > OK, this is finally workable, and it's clearly OMTC which landed around May > 20th. It's being worked on. It looks like you provided implicitly an answer to your question :-) Given a regression and a date we might be able to infer something. We can continue this discussion in Bug 1031032 if you like.