Perfherder has detected a awsy performance regression from push 9f39188a60d2c161a0deffe6fc885beed0706051. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

For more information on performance sheriffing please see our FAQ.

Set release status flags based on info from the regressing bug 1819128

Is there a way to generate profiles for these jobs? I added markers for update timers, so it should be easy to see what's going on if update timers are messing with the tests.

The behavior before bug 1819128 was that we would never fire more than one update timer at once. If two or more were scheduled to fire at the same time, we would fire one, then wait two minutes, fire the next one, etc... From a power use perspective, it's better to run everything at once.

Attempting to land the patch uncovered non-deterministic behavior in our tests, because our tests start with new profiles where no update timer has ever fired, so all of them are scheduled to fire for 30s after startup. Before the patch, that meant a first timer would fire after 30s, then we would have one firing every 2 minutes. After the patch, they all run at once 30s after startup. Some of these update timer callbacks do almost nothing and are very cheap, but some trigger a lot more work (triggering background updates that actually use CPU time). If the tests take more than 30s (likely given the "After tabs open [+30s, forced GC]" name), and some update timers are not disabled (very likely), it's likely that whatever happens in the parent process 30s after startup triggers IPCs to all content processes, and end up making the content process we are measuring use more memory.

(In reply to Florian Quèze [:florian] from comment #2)

Is there a way to generate profiles for these jobs?

Locally: MOZ_PROFILER_STARTUP=1 MOZ_PROFILER_SHUTDOWN="$(pwd)/profiles/prof-%p.json" ./mach awsy-test testing/awsy/awsy/test_base_memory_usage.py
And then samply load profiles/prof-78044.json gives me https://share.firefox.dev/3LVD4k5

At https://share.firefox.dev/3ndTsC7 we see there's update timer activity happening in the parent process, but it doesn't seem to send any IPC to preallocated content processes (I assume that's what we are measuring), thanks to bug 1817297.

Set release status flags based on info from the regressing bug 1819128

From REO Mtg: Will monitor for additional time in cycle to decide if this can be closed.

I don't see a way to make this actionable and I'm not sure there's an actual regression (vs a small change of what the test measures), so closing.