Open Bug 1779868 Opened 2 years ago Updated 2 years ago

Regression on AWFY around the July13 mark on speedometer-Elm-TodoMVC* benchmarks

Categories

(Core :: JavaScript Engine, defect, P3)

defect

Tracking

()

People

(Reporter: mayankleoboy1, Unassigned)

References

(Blocks 1 open bug)

Details

Go to AWFY : https://arewefastyet.com/win10/benchmarks/raptor-desktop-speedometer?numDays=60

Look at the following benchmarks :
Elm-TodoMVC/DeletingItems
Elm-TodoMVC/DeletingItems/Sync
Elm-TodoMVC
Elm-TodoMVC/Adding100Items/Sync
Elm-TodoMVC/CompletingAllItems/Sync
React-Redux-TodoMVC/CompletingAllItems/Sync
React-Redux-TodoMVC/DeletingItems

Unable to find a regression range.

Flags: needinfo?(afinder)

The regression range is:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=445cd004a4228df0a33898514edb3be98038b2fb&tochange=b363ae1f1a6d62e907818d48d616c53b494b40f0

Except that there is a merge of mozilla-central in the middle that I do not know how to interpret as it exhibit bi-modal behaviour and does not seems to show the slow down on the fast part.

If the regression does not exists before the merge, this would suggest that the regression is either coming from the merge commit or in the range:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=55ce1e6ba55cbb1436a3b7ecc5b5f8c9d629aa39&tochange=b363ae1f1a6d62e907818d48d616c53b494b40f0

Tom, Emilio, Jan, any idea what changes might have caused this regression?

Flags: needinfo?(jdemooij)
Flags: needinfo?(evilpies)
Flags: needinfo?(emilio)

My patch is an extension-API only patch, so unlikely to cause it I'd say.

Flags: needinfo?(emilio)

Bug 1779378 is a fix for a GC issue, but it's possible it changed GC timing, for example moving a GC to this particular test.

Looking at the Elm graph, however, it seems there was an improvement around the time bug 1529337 was backed out?

Flags: needinfo?(jdemooij)

Very interesting. Could we check if the tests use a Content-Security-Policy?

Flags: needinfo?(evilpies)

(In reply to Jan de Mooij [:jandem] from comment #3)

Looking at the Elm graph, however, it seems there was an improvement around the time bug 1529337 was backed out?

This does not seems to be the case with all Elm benchmarks.

Severity: -- → S4
Priority: -- → P3

There was an update to Speedometer 2.1 that landed at the end of May. I think it's unlikely that it caused the regressions, since those started appearing around July 13th. There is a similar issue reported on this topic that was closed as won't fix. There is also an upcoming ticket that will change how the Speedometer benchmark scores are extracted and computed, so that might change the results when it lands.

Flags: needinfo?(afinder)
You need to log in before you can comment on or make changes to this bug.