Joel Maher ( :jmaher ) (UTC -8) Reporter
	Description • 7 years ago

it appears that changing an operating system from windows 8 to windows 10 has presented some noisier data as well as less desirable numbers.  While this is not a regression in the sense that our firefox code changed it would be good to know if we are measuring something useful as compared to what our end users are seeing.

	Joel Maher ( :jmaher ) (UTC -8) Reporter
	Comment 1 • 7 years ago

for reference bug 1358306 is where we worked on getting windows 10 stood up, there are attachments there for about:support.

I have filed a bug to get loaners so I can get information.

	Joel Maher ( :jmaher ) (UTC -8) Reporter
	Comment 2 • 7 years ago

When I get the loaners I will get the graphics card version and post it here.

	Chris Peterson [:cpeterson]
	Comment 3 • 7 years ago

I'm attaching this bug to the Quantum release criteria bug so I don't lose it. Whether or why Windows 10 is slower than Windows 8 doesn't block Quantum, but it does affect how we interpret Talos results for the Quantum release criteria.

	Joel Maher ( :jmaher ) (UTC -8) Reporter
	Comment 4 • 7 years ago

this is the windows 8 about:support.

if I am reading this correctly, I see:
"adapterDescription": "NVIDIA GeForce GT 610 ",
"driverVersion": "9.18.13.3523",

	Joel Maher ( :jmaher ) (UTC -8) Reporter
	Comment 5 • 7 years ago

for windows 10 (https://bug1358306.bmoattachments.org/attachment.cgi?id=8870492 ):
"adapterDescription": "NVIDIA GeForce GT 610 ",
"driverVersion": "9.18.13.3523",


this looks to be the same driver for both systems; maybe the old driver doesn't work well on windows 10?

	Joel Maher ( :jmaher ) (UTC -8) Reporter
	Comment 6 • 7 years ago

as a note, the win10 stuff is from old data, I need to figure out how to get vnc to work on this loaner.

	Joel Maher ( :jmaher ) (UTC -8) Reporter
	Comment 7 • 7 years ago

I am still waiting on releng to get a win10 machine I can access and win8 machine I can get admin access to.

	Q
	Comment 8 • 7 years ago

The driver version reported in windows on ix machines is:

21.21.13.7878
2/23/2017

	Amy Rich [:arr] [:arich]
	Comment 9 • 7 years ago

The nvidia graphics driver for w10 got updated per the request from PI in https://bugzilla.mozilla.org/show_bug.cgi?id=1358307#c16

	Joel Maher ( :jmaher ) (UTC -8) Reporter
	Comment 10 • 7 years ago

win8: 9.18.13.3523
win10: 21.21.13.7878

:cpeterson- we have graphics info, the drivers are different, are there other things we should look at between the machines to help understand the differences in numbers?

	Chris Peterson [:cpeterson]
	Comment 11 • 7 years ago

Milan, can a gfx engineer take a look at this bug info to see if the different Nvidia driver versions might explain why Talos is so much slower on win10 than win8 on the same hardware? Are there other win10 graphics or DWM changes that might have caused this?

When Talos upgraded from win8 to win10, there was a clear discontinuity in the results for many tests, but particularly for tp5 [1] and tsvg_static [2]:

[1] https://treeherder.mozilla.org/perf.html#/graphs?timerange=2592000&series=%5Bmozilla-central,b68e2b084272409d7def3928a55baf0e00f3888a,1,1%5D&series=%5Bmozilla-central,ac46ba40f08bbbf209a6c34b8c054393bf222e67,1,1%5D&series=%5Bmozilla-central,c00763b23b39207671b795a12ba29d38ddc17f06,1,1%5D

[2] https://treeherder.mozilla.org/perf.html#/graphs?timerange=5184000&series=%5Bmozilla-central,397a484349ec684142dc3b3dab8f882a5d54bc8b,1,1%5D&series=%5Bmozilla-central,18cf40355e5b20164ab9307f83dd6d6eb6184aa8,1,1%5D&series=%5Bmozilla-central,e4e0081ff90530932c463fc917d113936690baa3,1,1%5D

	Milan Sreckovic [:milan] (needinfo for best results)
	Comment 12 • 7 years ago

(In reply to Chris Peterson [:cpeterson] from comment #11)
> Milan, can a gfx engineer take a look at this bug info to see if the
> different Nvidia driver versions might explain why Talos is so much slower
> on win10 than win8 on the same hardware? Are there other win10 graphics or
> DWM changes that might have caused this?

There were a lot of changes between Win 8 and Win 10 :)

We can see if updating Windows 8 drivers from 335.23 to 378.78 changes the results, but I have a feeling that it's probably a wider system issue.

Can we capture profiles on the two systems for a (random) test that shows the worst regression?

	Joel Maher ( :jmaher ) (UTC -8) Reporter
	Comment 13 • 7 years ago

:mconley- do you have some docs for getting profiling to work from talos?  I know it is problematic and you got it working on try server recently.

	Mike Conley (:mconley) (:⚙️)
	Comment 14 • 7 years ago

(In reply to Joel Maher ( :jmaher) from comment #13)
> :mconley- do you have some docs for getting profiling to work from talos?  I
> know it is problematic and you got it working on try server recently.

I _believe_ the main problem is that you have to request more than one rebuild with --rebuild-talos > 1 (bug 1370907). The other thing is that a number of the Talos tests are using the old Profiler.js script, which means that they'll sometimes only report a single processes samples, which isn't always what you want.

Also, sometimes symbolication just kinda times out in automation. Sometimes, when you can get profiles from Talos, it's kind of a miracle.

cpeterson, you mentioned tp5 and tsvgx - were there other tests in that list that showed a wide difference between Win 8 and Win 10? I ask because some might be easier than others to get profiles from.

	Chris Peterson [:cpeterson]
	Comment 15 • 7 years ago

(In reply to Mike Conley (:mconley) from comment #14)
> cpeterson, you mentioned tp5 and tsvgx - were there other tests in that list
> that showed a wide difference between Win 8 and Win 10? I ask because some
> might be easier than others to get profiles from.

There are discontinuities in most of the other Talos tests, including ts_paint, tpaint, and tart as shown on the dashboards below:

https://health.graphics/quantum/

	Mike Conley (:mconley) (:⚙️)
	Comment 16 • 7 years ago

Okay. ts_paint was made to work with profiling very recently in bug 1372361. That's one we can / should gather a profile from from both types of machines for comparison.

	Joel Maher ( :jmaher ) (UTC -8) Reporter
	Comment 17 • 6 years ago

windows 8 is not easily accessible anymore- lets move forward.