Closed
Bug 771745
Opened 12 years ago
Closed 12 years ago
Compare startup(firstPaint, firstloaduri) for different values of STARTUP_USING_PRELOAD_TRIAL
Categories
(Mozilla Metrics :: Data/Backend Reports, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
Moved to JIRA
People
(Reporter: taras.mozilla, Unassigned)
References
Details
(Whiteboard: [Telemetry:P1])
Attachments
(1 file)
(deleted),
application/pdf
|
Details |
+++ This bug was initially created as a clone of Bug #765850 +++
Saptashi, we need to redo this analysis filtering out warm startups.
I have one more theory, Saptashi, can you filter the data by SIMPLE_MEASURES_START?
According to https://metrics.mozilla.com/data/content/pentaho-cdf-dd/Render?solution=metrics2&path=%2Ftelemetry&file=telemetryHistogram.wcdf&bookmarkState={%22impl%22%3A%22client%22%2C%22params%22%3A{%22startDate%22%3A%222012-05-29%22%2C%22endDate%22%3A%222012-06-27%22%2C%22appVersion%22%3A%22%22%2C%22appName%22%3A%22Firefox%22%2C%22arch%22%3A%22%22%2C%22OS%22%3A%22%22%2C%22version%22%3A%22%22%2C%22channel%22%3A%22nightly%22%2C%22reason%22%3A%22idle-daily%22%2C%22appBuildID%22%3A%22%22%2C%22fromPlatformBuildID%22%3A%22%22%2C%22toPlatformBuildID%22%3A%22%22%2C%22excludeParam%22%3A%22%22%2C%22measure%22%3A%22SIMPLE_MEASURES_START%22%2C%22histogramCompareParam%22%3A%22appVersion%22%2C%22histogramVariablesParam%22%3A%22%22%2C%22platformBuildIDMode%22%3A%22LATEST%22%2C%22platformBuildIDTopCount%22%3A%2230%22%2C%22conditionsStatistic%22%3A%22%23conditionsStatistic%22%2C%22submissionsParameter%22%3A[[59459]]}}
23% of our users take >=1second before firefox code executes...Those could be 23% that do slow & cold startups. Could you redo the analysis, but filter on simpleMeasures.start > 0? I'm wondering if the fact that most startups are warm is skewing the data.
I'd also like to know the distribution of simpleMeasures.start for each value of STARTUP_USING_PRELOAD_TRIAL.
Comment 1•12 years ago
|
||
Status: NEW → ASSIGNED
Target Milestone: Unreviewed → Targeted - JIRA
Comment 2•12 years ago
|
||
Btw, there were 48,762 records for the following conditions
json$info$appName == 'Firefox'
&& json$info$OS == 'WINNT'
&& json$info$appUpdateChannel == "nightly
&& appBuildId in (20120617 ... 20120623)
&& dateofsubmission from 20120617 ... 20120623
The quartiles of START is (raw units , i guess ms)
0% 25% 50% 75% 100%
2 31 127 742 418823980
So all starts > 0.
Better someone explains what the role of start is in this analysis?
Comment 3•12 years ago
|
||
Erm, slight modification. 4% of the data have start == 0. I'll complete this soon.
Definition of start still pending.
Comment 4•12 years ago
|
||
So as mentioned, only 4% of records have start == 0. The next higher value is 2 (see Comment 2).
With a percentage so small and a delta (2ms) so small, i would expect little difference in the analysis.
Indeed, there isn't.
As for distribution of start for prefetch X preload (where start is now in seconds)
$preloadOff.prefetchOff
0% 5% 25% 50% 75% 98% 100%
0.003 0.015 0.035 0.123 0.452 11.306 750.999
$preloadON.prefetchOff
0% 5% 25% 50% 75% 98% 100%
0.003 0.015 0.039 0.125 0.484 13.245 360.605
$preloadOff.prefetchON
0% 5% 25% 50% 75% 98% 100%
0.002 0.010 0.031 0.140 0.991 15.938 881.878
$preloadON.prefetchON
0% 5% 25% 50% 75% 98% 100%
0.002 0.010 0.031 0.126 1.007 16.691 581.385
Comment 5•12 years ago
|
||
In case the table above is difficult to decipher the figure will help.
The figure has percentiles of Log(start,2) (2%,25%,50,75,and 98). As can be seen, Prefetch = On is associated with larger values of start as compared to prefetch=Off.
Also, no interaction with prefetch and preload.
Comment 6•12 years ago
|
||
If I'm reading the data correctly, I think it says that cold startups are positively affected pretty significantly when prefetch is disabled.
But there is no way to disable prefetch only for some startups, nor can we disable it only for a specific firefox install only.
Since most installs are hot startups, I think the backout process for disabling prefetch should continue. Do you agree Taras?
Reporter | ||
Comment 7•12 years ago
|
||
(In reply to Brian R. Bondy [:bbondy] from comment #6)
> Since most installs are hot startups, I think the backout process for
> disabling prefetch should continue. Do you agree Taras?
I think we should turn off prefetch when START > 0.5s. For safety we can turn it back on on an interval
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Reporter | ||
Comment 8•12 years ago
|
||
Sorry, didn't mean to close this.
Saptashi, the data in comment 4 is very handy, however we need a matching table for firstPaint/firstLoadURI to make a decision.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment 9•12 years ago
|
||
For firstpaint (seconds)
$prefetchOff.preloadOff
2% 25% 50% 75% 98%
0.57900 2.07050 5.82000 14.50750 80.03664
$prefetchON.preloadOff
2% 25% 50% 75% 98%
0.53100 1.56450 3.79600 10.59325 67.91190
$prefetchOff.preloadON
2% 25% 50% 75% 98%
0.64000 2.24750 5.48300 13.21050 79.81156
$prefetchON.preloadON
2% 25% 50% 75% 98%
0.54924 1.60800 3.65200 9.09600 64.25872
For *URI (seconds)
$prefetchOff.preloadOff
2% 25% 50% 75% 98%
0.44100 1.91200 5.97600 14.79150 85.87704
$prefetchON.preloadOff
2% 25% 50% 75% 98%
0.39022 1.37575 3.96100 11.04075 72.41188
$prefetchOff.preloadON
2% 25% 50% 75% 98%
0.4860 2.0910 5.5940 13.6750 82.8168
$prefetchON.preloadON
2% 25% 50% 75% 98%
0.39100 1.45200 3.77600 9.86000 69.57224
Reporter | ||
Comment 10•12 years ago
|
||
According to the 48K number above populations for last 2 columns are ~12K, 2K...So these wouldn't be swayed by a few outliers.
Looks like priming prefetch with preload is the best way to go(ie least risk of regressing people). Thanks a lot Saptashi!
Status: REOPENED → RESOLVED
Closed: 12 years ago → 12 years ago
Resolution: --- → FIXED
Comment 11•12 years ago
|
||
Great data :)
OK so the plan is to re-enable preload on Windows (always) and continue with reverting the disable prefetch task (There is only 1 patch left to land in 2 weeks). Correct?
Reporter | ||
Comment 12•12 years ago
|
||
(In reply to Brian R. Bondy [:bbondy] from comment #11)
> OK so the plan is to re-enable preload on Windows (always) and continue with
> reverting the disable prefetch task (There is only 1 patch left to land in 2
> weeks). Correct?
I think that's the best short-term action based on this.
You need to log in
before you can comment on or make changes to this bug.
Description
•