Central-as-beta simulation

How to run these simulations

Failure log:

[task 2022-08-25T11:14:39.323Z] 11:14:39     INFO - TEST-PASS | browser/base/content/test/performance/browser_startup_images.js | Loaded image chrome://global/skin/icons/pocket-outline.svg should have been shown. - 
[task 2022-08-25T11:14:39.324Z] 11:14:39     INFO - Buffered messages finished
[task 2022-08-25T11:14:39.327Z] 11:14:39     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/performance/browser_startup_images.js | Loaded image chrome://branding/content/icon32.png should have been shown. - 
[task 2022-08-25T11:14:39.328Z] 11:14:39     INFO - Stack trace:
[task 2022-08-25T11:14:39.329Z] 11:14:39     INFO - chrome://mochikit/content/browser-test.js:test_ok:1440
[task 2022-08-25T11:14:39.330Z] 11:14:39     INFO - chrome://mochitests/content/browser/browser/base/content/test/performance/browser_startup_images.js:null:120
[task 2022-08-25T11:14:39.339Z] 11:14:39     INFO - TEST-PASS | browser/base/content/test/performance/browser_startup_images.js | Loaded image chrome://browser/content/static-robot.png should have been shown. - 

Hello Scott, could you please take a look at this failure? It could be regressed by Bug 1782318. Thanks

I'm not sure how bug 1782318 could have caused this. bug 1782318 is happening almost exclusively in a preffed off feature we are going to experiment with.

Unless the test is somehow setting up images with no image source.

Either way I'll investigate because I've been surprised before with tests.

Is there a way I can get this test running locally? If I run it locally, it passes, so I suspect I am missing proper parameters.

It only runs for debug (1, 2).

If it passes for debug for you, could you verify the central-as-beta config has been activated?

Is there a way I can run that with ./mach test, and not ./mach try? That way I can debug it if needed?

I'm not sure what to do with central-as-beta config or setting debug to true, but I'll keep looking.

So I got it running with debug true, via mozconfig.

I am still looking into what central-as-beta config is.

So far with just debug, I get the same errors before and after the patch in bug 1782318.

Is central-as-beta essentially applying the patch to beta, then running the tests?

If you want to check that your patch works on a central-as-beta simulation, make sure your patch is imported and then run ./mach try release -v 106.0b1 --tasks release-sim --migration central-to-beta If not all the jobs shall run (e.g. to verify a test fix), append --stage-changes to the command, hg commit -m "Early beta sim" and select the jobs with ./mach try chooser --no-artifact.

(In reply to Scott [:thecount] Downe from comment #8)

Is central-as-beta essentially applying the patch to beta, then running the tests?

The Central as Beta simulation tests the recent nightly changes in order to prepare it for a Beta merge

I ran a bunch of try builds with that command, and it did fail once, but I am unsure if that is significant.

I'm trying to get some help on this as I have no clue what to try next or how this might be connected.

If you have any other ideas as to what else this might be, it might be worth investigating other possibilities.

These three try runs using the above command with only my commit on top, and nothing after it, show no fails. Could it be another commit that happened after mine that caused these failures?

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ec16736342e29f69cf60a168148b9363ccfa1a64
https://treeherder.mozilla.org/#/jobs?repo=try&revision=3b433752ffdf5e5a1a20b6458814f87404fc75e4
https://treeherder.mozilla.org/#/jobs?repo=try&revision=43d81bed3b9ee1331a1214545b499994a5e53b78

Hi Scott!
Atm Aryx is investigating which change regressed this.

It's not from one of these: bug 1782318, bug 1771311, bug 1777198. Will investigate further.

This is a regression from bug 1786293 and is reproducible with debug builds in central-as-beta configuration.

I suspect the solution is likely going to be change the pref value for beta.

(In reply to :Gijs (he/him) from comment #16)

I suspect the solution is likely going to be change the pref value for beta.

Yeah.

(In reply to Dão Gottwald [::dao] from comment #17)

(In reply to :Gijs (he/him) from comment #16)

I suspect the solution is likely going to be change the pref value for beta.

Yeah.

I'm not clear on which pref needs to be changed to what value to get this to pass? Can you elaborate on the next steps needed here :dao?

(In reply to Sam Foster [:sfoster] (he/him) from comment #19)

(In reply to Dão Gottwald [::dao] from comment #17)

(In reply to :Gijs (he/him) from comment #16)

I suspect the solution is likely going to be change the pref value for beta.

Yeah.

I'm not clear on which pref needs to be changed to what value to get this to pass? Can you elaborate on the next steps needed here :dao?

See the above patch :)

https://hg.mozilla.org/mozilla-central/rev/cb5977c8af25

Verified fixed in today's central-as-beta simulation