Closed Bug 1680727 Opened 4 years ago Closed 4 years ago

3.28 - 108.5% allrecipes / allrecipes fcp / allrecipes loadtime / amazon-search / amazon-search fcp / amazon-search loadtime (android-hw-g5-7-0-arm7-api-16-shippable) regression on push 741ea917c5e87c86e37c43f2a4c92858df8fee7b (Wed December 2 2020)

Categories

(Testing :: Raptor, defect)

Product:
Testing
Component:
Raptor
Version:
Firefox 85
Type:
defect

Tracking

(firefox85 affected)

Status:
RESOLVED WONTFIX
Tracking Flags:
Tracking Status
firefox85 --- affected

People

(Reporter: Bebe, Unassigned)

References

(Regression)

Details

(Keywords: perf, perf-alert, regression)

Perfherder has detected a browsertime performance regression from push 741ea917c5e87c86e37c43f2a4c92858df8fee7b. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio Suite Test Platform Options Absolute values (old vs new)
109% allrecipes fcp android-hw-g5-7-0-arm7-api-16-shippable nocondprof warm 416.25 -> 867.88
108% allrecipes fcp android-hw-g5-7-0-arm7-api-16-shippable nocondprof warm 419.38 -> 871.71
27% allrecipes android-hw-g5-7-0-arm7-api-16-shippable nocondprof warm 1,033.11 -> 1,308.11
7% amazon-search fcp android-hw-g5-7-0-arm7-api-16-shippable nocondprof warm 826.23 -> 881.21
6% amazon-search android-hw-g5-7-0-arm7-api-16-shippable nocondprof warm 983.80 -> 1,041.29
5% allrecipes loadtime android-hw-g5-7-0-arm7-api-16-shippable nocondprof warm 4,397.83 -> 4,607.46
3% amazon-search loadtime android-hw-g5-7-0-arm7-api-16-shippable nocondprof warm 1,666.06 -> 1,720.79

Improvements:

Ratio Suite Test Platform Options Absolute values (old vs new)
9% google android-hw-g5-7-0-arm7-api-16-shippable nocondprof warm webrender 242.08 -> 220.49
9% google loadtime android-hw-g5-7-0-arm7-api-16-shippable nocondprof warm webrender 242.83 -> 221.54
9% google fcp android-hw-g5-7-0-arm7-api-16-shippable nocondprof warm webrender 236.38 -> 215.79

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Component: Performance → Raptor
Flags: needinfo?(gmierz2)

It's really interesting that it only regressed the warm pageload. We'll accept these as they are more realistic and I don't see any issues in the videos either. cc'ed :acreskey in case he's interested.

Flags: needinfo?(gmierz2)

That is interesting.
I'm also surprised that more tests were not affected by this in general.

:acreskey, I'm going to close this as wontfix. If you think we should look into this some more though feel free to file a bug about it and we can discuss further there.

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WONTFIX

I agree with closing as wontfix - this change is a big step towards capturing a more representative codepath.

You need to log in before you can comment on or make changes to this bug.