Closed
Bug 1059349
Opened 10 years ago
Closed 9 years ago
[Calendar] Startup time regressions of ~100 ms from 2.0 to 2.1
Categories
(Firefox OS Graveyard :: Gaia::Calendar, defect)
Tracking
(Not tracked)
RESOLVED
WORKSFORME
People
(Reporter: tchung, Assigned: gaye)
References
Details
(Keywords: perf, regression, Whiteboard: [priority])
Attachments
(1 file)
(deleted),
application/zip
|
Details |
[Blocking Requested - why for this release]:
Per Datazilla, Calendar cold startup numbers have regressed about 100 ms since 2.0 to 2.1.
Environment: startup_>_moz-app-visually-complete, flame-319MB, workload = light. human perception goal = 1000ms
* 2.0 (All numbers use median, looking at latest numbers from 8-20 to 8-25)
** https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar&app=calendar&gaia_rev=d72f8ad53448aed5&gecko_rev=2a18149b3ae8&plot=avg
calendar -- High, 1203; Low, 1131; Trend ~1160
* 2.1 (All numbers use median, looking at numbers from 8-08 to 8-13 (last available))
* https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=30&test=startup_%3E_moz-app-visually-complete&app_list=calendar&app=calendar&gaia_rev=a2219a55145e730e&gecko_rev=97a5d21e57ac&plot=avg
calendar -- High, 1318; Low, 1219; Trend ~1260
Please investigate, thanks.
Comment 1•10 years ago
|
||
blocking based on performance regression. Team to investigate
blocking-b2g: 2.1? → 2.1+
Updated•10 years ago
|
Assignee: nobody → mmedeiros
Target Milestone: --- → 2.1 S4 (12sep)
Comment 2•10 years ago
|
||
I really don't think the perf regression was introduced by the calendar app itself, specially since we have regressions on all the productivity apps.
by looking only at the datazilla graph it looks like perf regression was introduced mainly by https://github.com/mozilla-b2g/gaia/commit/62eedafb0657bbec (mozl10n, which affects multiple apps)
there was some changes on the perf graph before that (a few ms), [this commit](https://github.com/mozilla-b2g/gaia/commit/6cc134a8d8b19ee2) also change a little bit how we are measuring the performance and happened at the same day. - so maybe perf did not change, we are only measuring it differently...
and a few other commits that affected the calendar app landed on 2014/07/23 and 2014/07/24: https://github.com/mozilla-b2g/gaia/commit/3078ac2eb (small CSS change, probably unrelated) and https://github.com/mozilla-b2g/gaia/commit/500ebd79331 (theme color, affects many apps at once)
whatever happened, probably landed between July 23 and 24th.
Updated•10 years ago
|
Comment 3•10 years ago
|
||
I found a few weird commits between https://github.com/mozilla-b2g/gaia/compare/9dd4bf0...9f4ad73
I'm almost sure that the main regression was caused by commit https://github.com/mozilla-b2g/gaia/commit/9f4ad7301db7b3a034f012f0af4eb97df06c818e (Bug 1042083) but there are also some other regressions and changes in the way performance events are measured.
Here are the average results for 10 runs on my Flame running Gecko 34.0a1 (https://hg.mozilla.org/mozilla-central/rev/6f702709fab6)
# 9f4ad73 (perf regression to startup & dom-loaded)
## Bug 1042083 - Implement App Titlebar
startup: 832.7
moz-chrome-dom-loaded: 1324.4761820000017
# b9240ad (small regression to startup & dom-loaded)
## Bug 1041985 - [Search] Enable places
startup: 713.5
moz-chrome-dom-loaded: 1170.5350103
# 6cc134a (dom-loaded stabilized here)
## Bug 1037520 - Include start time capture in the repeat phase, not test start phase
startup: 685.9
moz-chrome-dom-loaded: 1071.0687652999998
# 9dd4bf0 (dom-loaded is way lower than following commits)
## Bug 1030352 - Delete/undo-delete phone number button in edit dialog does not have an accessibility label
startup: 661.3
moz-chrome-dom-loaded: 823.6056771000007
Comment 4•10 years ago
|
||
Performance results for Gaia commits 9f4ad73, b9240ad, 6cc134a & 9dd4bf0 running on:
Device Flame
Gecko https://hg.mozilla.org/mozilla-central/rev/6f702709fab6
BuildID 20140722040212
Version 34.0a1
RUNS=10 MARIONETTE_RUNNER_HOST=marionette-device-host APP=calendar make test-perf
Comment 5•10 years ago
|
||
and it looks like the performance regression introduced by commit 9f4ad73 was detected and "fixed" by Bug 1043857 a few days later (did not test it yet). I'll keep investigating it tomorrow, but this is a very SLOW (and boring) process..
Comment 6•10 years ago
|
||
by analyzing the data from datazilla (https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=30&test=startup_%3E_moz-app-visually-complete&app_list=calendar&app=calendar&gaia_rev=a2219a55145e730e&gecko_rev=97a5d21e57ac&plot=avg) we can see that the commit https://github.com/mozilla-b2g/gaia/commit/8532cb66d11048c8 (Bug 1043857) reduced visually complete by around 150ms but still 100ms above the old value before 9f4ad73..
we can also see that between https://github.com/mozilla-b2g/gaia/compare/7306065260ceabd3...5c90a723bb824228 performance degraded around 100ms (no info between these points)
so I would say that multiple features are affecting the overall performance. I'll try to dig more info about it.
Comment 7•10 years ago
|
||
the perf regression between https://github.com/mozilla-b2g/gaia/compare/7306065260ceabd3...5c90a723bb824228 was caused by https://github.com/mozilla-b2g/gaia/commit/166a35556ef4e4dd48a91a4773a1bec674869e55 (Bug 1042422) - we changed the way month view works to actually wait all the calendars to be loaded before rendering the view (which adds ~100ms delay). The thing is that 2.0 also contains this patch (and performance was also affected by it).
The difference between moz-chrome-dom-loaded & moz-app-visually-complete before the patch was ~100ms, after the patch it takes ~200ms.
My investigation is complete, now we need to decide how it is going to be fixed and if we need to address it.
Reporter | ||
Comment 8•10 years ago
|
||
looping in product. we need to understand what is the requirement what the Calendar app will need to run at to maintain an acceptable baseline performance for 2.1. We also need requirements defining what the acceptable amount of time it is before we are alerting an unacceptable performance.
FYI, the 2.0 numbers show ~1160ms. so is +30ms over acceptable? +50ms? +100ms?
Flags: needinfo?(skasetti)
Comment 9•10 years ago
|
||
Fwiw - we have said in the past that we have a flat 1,000ms threshold for launch time across all apps. I'm not sure if that's changed recently or not. Looping in Eli as well because he probably has the most information here.
Comment 10•10 years ago
|
||
The accepted-upon performance criteria [1] for application cold-launch times was determined to be 1000ms for each application. As far as regression determination, we generally would do investigation for anything that caused an increase of at least 30ms~50ms.
[1] https://wiki.mozilla.org/FirefoxOS/Performance/Release_Acceptance
Comment 11•10 years ago
|
||
can we confirm if the change to Rocketbar/Places (Bug 1063789) resolved the performance problem or if we still need to do anything else? We don't have 2.1 branch on datazilla yet..
Keywords: qawanted
Reporter | ||
Comment 12•10 years ago
|
||
(In reply to Miller Medeiros [:millermedeiros] from comment #11)
> can we confirm if the change to Rocketbar/Places (Bug 1063789) resolved the
> performance problem or if we still need to do anything else? We don't have
> 2.1 branch on datazilla yet..
2.1 branch on datazilla is up at: https://datazilla.mozilla.org/b2g/?branch=v2.1&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar,camera,clock,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,music,settings,sms,video&app=calendar&plot=avg
but for some reasons i cant see calendar app. was this removed by chance?
Updated•10 years ago
|
Target Milestone: 2.1 S4 (12sep) → 2.1 S5 (26sep)
Comment 13•10 years ago
|
||
unassigned since commits that introduced regression aren't part of the calendar app (comment #6) and I really don't know what to do in this case.
Assignee: mmedeiros → nobody
Target Milestone: 2.1 S5 (26sep) → ---
Comment 14•10 years ago
|
||
(In reply to Miller Medeiros [:millermedeiros] from comment #11)
> can we confirm if the change to Rocketbar/Places (Bug 1063789) resolved the
> performance problem or if we still need to do anything else? We don't have
> 2.1 branch on datazilla yet..
Results:
Device: Flame 2.1
Build ID: 20140926081140
Gaia: df567808f51a8c7c33b3651542c67a277940d04d
Gecko: 3e3c69b549b0
Version: 34.0a2
Firmware Version: v180
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0
STR:
1. Tap Rocket Bar and search for desired app.
2. Select app.
3. Allow app to open and observe "Time to Load" output.
Average Time to Load outputs observed (10 repro averages):
Rocket Bar search (319 MB) to:
Calendar app ~850ms
Contacts app ~1126ms
Settings app ~1316ms
Rocket Bar search (512 MB) to:
Calendar app ~850ms
Contacts app ~1037ms
Settings app ~1264ms
Updated•10 years ago
|
QA Whiteboard: [QAnalyst-Triage?] → [QAnalyst-Triage+]
Flags: needinfo?(jmitchell)
Time to Load in the HUD isn't equivalent to the Datazilla startup tests. It only measures to first paint, not until complete above the fold visual composition.
Note that 2.1 is in Datazilla on its own branch at this point. Devs can also test any local changes at their desk using make test-perf per https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_performance_tests
Updated•10 years ago
|
Assignee: nobody → gaye
Target Milestone: --- → 2.1 S6 (10oct)
Comment 17•10 years ago
|
||
Team confirmed this will not block but high-priority for backlog
blocking-b2g: 2.1+ → backlog
Whiteboard: [priority]
Updated•10 years ago
|
Target Milestone: 2.1 S6 (10oct) → ---
Updated•10 years ago
|
Flags: needinfo?(skasetti)
Updated•10 years ago
|
blocking-b2g: backlog → ---
Updated•10 years ago
|
tracking-b2g:
+ → ---
Comment 19•9 years ago
|
||
worksforme for v2.1. Improvement bug 1181018 has been created in v2.5.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WORKSFORME
Updated•9 years ago
|
You need to log in
before you can comment on or make changes to this bug.
Description
•