Ryan VanderMeulen [:RyanVM] Reporter
	Description • 9 years ago

Talos e10s currently only runs on m-c. As we discovered today, that makes them highly-prone to unexpected bustage that only shows up on m-c. Per the visibility policy, I've hidden them until they are running on all required trees.

	Joel Maher ( :jmaher ) (UTC -8)
	Comment 1 • 9 years ago

retriggers a pgo build to bisect the list of merges:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=talos linux svgr-e10s&exclusion_state=all&fromchange=759ee85bf3f0&tochange=7c5f187b65bf

this is running but failing on CART.


Regarding the visibility policy, I would like to run talos e10s on all pgo builders instead of just m-c.  That would make this visible and go on our path to increase coverage.

mconley- you are familar with tart as well as e10s, any idea what landed in the last 24 hours which would cause e10s to fail (linux*, win*) ?

	Mike Conley (:mconley) (:⚙️)
	Comment 2 • 9 years ago

A number of stability fixes have landed recently to reduce the number of tab crashes.

How can I get more detail about what kind of bustage talos is hitting? What stage does it get to before it breaks?

	Joel Maher ( :jmaher ) (UTC -8)
	Comment 3 • 9 years ago

the logs are not saying a whole lot, but I did run it locally.

I was able to reproduce it with a downloaded pgo build and a regular opt build (linux64) from my local tree.

I fail on tart and this is the end of my output:
iconFade-open-DPI2.half.TART: 2.1
iconFade-open-DPI2.all.TART: 2.2
iconFade-open-DPI2.error.TART: 2.8
newtab-open-preload-no.raw.TART: 22.0 3.3 9.1 1.3 4.7 1.7 2.9 2.0 4.8 6.1 2.9 15.5 22.5 21.1 14.2 9.1 12.3 11.5 17.3 39.8 4.5 13.8 3.9 6.2 6.0
newtab-open-preload-no.half.TART: 12.8
newtab-open-preload-no.all.TART: 10.3
newtab-open-preload-no.error.TART: 31.8
newtab-open-preload-yes.raw.TART: 15.3 18.1 6.7 9.6 3.8 4.5 5.1 4.0 4.5 5.7 5.8 5.0 5.4 5.0 4.8 5.4 5.4 4.9 5.2 6.3 4.9 5.0 3.5 3.0 2.5 2.5 2.4 2.5 3.1 2.7 2.9 5.6 5.9 5.8 2.8 2.1 2.2 3.1 2.8 2.7 2.6 2.7 3.0 3.4 3.0 6.3 2.8 2.8 3.0 2.2 2.2 2.1 2.4 2.1 3.7 2.8 2.6 2.5 3.9 4.5 3.2 2.9
newtab-open-preload-yes.half.TART: 3.1
newtab-open-preload-yes.all.TART: 4.3
newtab-open-preload-yes.error.TART: 47.9


no matter what I do, it always fails at this spot in the logs.  without --e10s it works just fine.  

:mconley, do let me know if you need more information.

	Mike Conley (:mconley) (:⚙️)
	Comment 4 • 9 years ago

Does this sound related to what you were looking at, handyman?

	David Parks [:handyman]
	Comment 5 • 9 years ago

Nope, bug 1124728 is a mac thing.

	Mike Conley (:mconley) (:⚙️)
	Comment 6 • 9 years ago

What state is the browser in at the moment it fails? Does it look like a tab has crashed?

Also, off chance, does that forceInitialBrowserRemote thing address this?

	Joel Maher ( :jmaher ) (UTC -8)
	Comment 7 • 9 years ago

I don't have an OSX machine, so I couldn't tell you for sure.  From looking at the information in the logs from the job, it looks as though we launch the browser on the warmup run and never close it.  We will see in a day or so if the forceInitialBrowserRemote addresses it.  This has been broken long before it was broken on linux/windows.

	Mike Conley (:mconley) (:⚙️)
	Comment 8 • 9 years ago

Hey Joel - the description of this bug, and comment 3, seem to describe two different actions (description is about unhiding, comment 3 is to fix things to make unhiding possible).

Would it be possible for you to file a new bug for fixing the Talos lock-up issues with e10s (if there isn't one already), and mark that blocking this, and then nom that for tracking-e10s?

	Joel Maher ( :jmaher ) (UTC -8)
	Comment 9 • 9 years ago

associated 3 bugs with this, it should track all the known issues related to non intermittent talos failures on e10s!