0.72 - 4.28% cnn PerceptualSpeedIndex / cnn SpeedIndex / ebay-kleinanzeigen-search LastVisualChange / ebay-kleinanzeigen-search fcp (android-hw-g5-7-0-arm7-api-16-shippable) regression on push 4bce303c45e740a5636fa88edfbbe41b6cd2e35e (Thu August 27 2020)
Categories
(Testing :: geckodriver, defect)
Tracking
(firefox-esr68 unaffected, firefox-esr78 unaffected, firefox80 unaffected, firefox81 unaffected, firefox82 affected)
Tracking | Status | |
---|---|---|
firefox-esr68 | --- | unaffected |
firefox-esr78 | --- | unaffected |
firefox80 | --- | unaffected |
firefox81 | --- | unaffected |
firefox82 | --- | affected |
People
(Reporter: Bebe, Unassigned)
References
(Regression)
Details
(Keywords: perf, perf-alert, regression)
Perfherder has detected a browsertime performance regression from push 4bce303c45e740a5636fa88edfbbe41b6cd2e35e. As author of one of the patches included in that push, we need your help to address this regression.
Regressions:
4% ebay-kleinanzeigen-search LastVisualChange android-hw-g5-7-0-arm7-api-16-shippable opt cold 3,930.48 -> 4,098.83
4% cnn PerceptualSpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 4,818.87 -> 5,002.00
3% cnn SpeedIndex android-hw-g5-7-0-arm7-api-16-shippable opt cold 5,377.43 -> 5,562.17
1% ebay-kleinanzeigen-search fcp android-hw-g5-7-0-arm7-api-16-shippable opt cold 1,231.02 -> 1,239.83
Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.
For more information on performance sheriffing please see our FAQ.
Reporter | ||
Updated•4 years ago
|
Comment 1•4 years ago
|
||
This change cannot make any difference. The pref isn't used for a while in Firefox and even if it would, it's only related to Firefox upgrades. Are you sure that you have the correct range or that the numbers are correct?
Reporter | ||
Comment 2•4 years ago
|
||
Did some re triggers and backfills to have a better confirmation.
leaving the NI? open
Comment 3•4 years ago
|
||
Set release status flags based on info from the regressing bug 1583504
Comment 4•4 years ago
|
||
Florin, any update here?
Reporter | ||
Comment 5•4 years ago
|
||
the graph confirms this is he commit that caused the regression
Comment 6•4 years ago
|
||
In that case there is clearly something wrong with the performance tests. As I said the removed code isn't in use for a long time, and on central there is no single use of it anymore:
https://searchfox.org/mozilla-central/search?q=extensions.showMismatchUI&path=
Greg, any idea what this could be?
Comment 7•4 years ago
|
||
It looks like the regression detection failed here and that your patch didn't cause the regression here. Have a look at this graph: https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&series=autoland,2473343,1,13&timerange=2592000&zoom=1597693949000,1600266039250,876.2777777777778,1625.7222222222222
From there I think it's clear that a patch on August 26th caused the regression, not your patch. :bebe can you bisect further to get the true culprit?
Reporter | ||
Updated•4 years ago
|
Reporter | ||
Comment 8•4 years ago
|
||
you are right Opened new bug:
Bug 1667974 - 0.72 - 4.53% cnn PerceptualSpeedIndex / cnn SpeedIndex / ebay-kleinanzeigen-search LastVisualChange / ebay-kleinanzeigen-search fcp (android-hw-g5-7-0-arm7-api-16-shippable) regression on push f8cda577ac2f0eb74b5707f5f8140d656994628f (Wed August 26 2020)
Description
•