Closed Bug 1082268 Opened 10 years ago Closed 9 years ago

[meta] Music app launch latency is worse compared to Android on quad core 1 GB device

Categories

(Firefox OS Graveyard :: Gaia::Music, defect, P1)

Product:
Firefox OS Graveyard
Component:
Gaia::Music
Platform:
ARM
Gonk (Firefox OS)
Type:
defect

Tracking

(tracking-b2g:-)

Status:
RESOLVED WORKSFORME
Project Flags:
tracking-b2g -

People

(Reporter: tkundu, Assigned: bobby.chien+bugzilla)

References

Details

(Keywords: perf, Whiteboard: [caf priority: p2][CR 786374]) 

We tried with fix from bug 1069452 and we are still seeing 1406ms for Music app launch latency on msm8226(4 cores and 1GB RAM) with following gaia/gecko in v2.1. 

https://www.codeaurora.org/cgit/quic/lf/b2g/mozilla/gaia/commit/?h=mozilla/v2.1&id=9861c61ec302fb0316c753a2e1c0f592180515fd
https://www.codeaurora.org/cgit/quic/lf/b2g/mozilla/gecko/commit/?h=mozilla/v2.1&id=81d98fe40d10d5a8a48664aa60913491db215a4f

I also saw that Mozilla is also hitting 1237ms for Music launch latency in 
https://wiki.mozilla.org/B2G/QA/2014-10-02_Performance_Acceptance#Music

Please note that we didn't see any significant gain in launch latency when we moved from dual core(msm8610) to four core(msm8226). 

Please also look into bug 1074783 comment 19 and bug 1082262

But Android is showing 829ms for music app launch latency on same platform.

We need to improve Music app launch latency to match with Android on same platform
blocking-b2g: --- → 2.2?
(In reply to Tapas[:tkundu on #b2g/gaia/memshrink/gfx] (always NI me) from comment #0)

> I also saw that Mozilla is also hitting 1237ms for Music launch latency in 
> https://wiki.mozilla.org/B2G/QA/2014-10-02_Performance_Acceptance#Music

Tapas,

The number you quote here (and elsewhere in similar bugs referencing the acceptance study) is the p95 number--that is, the 95th percentile load time (95% of all loads faster than this).

The median load time for music as determined in this study is 1093 ms.

Note also that those results were bimodal, due to a measurement issue in the test harness. A later acceptance study on Oct 20's nightly build (and with harness fixed) showed a median load time of 845 ms. That's almost comparable to Android's.

https://wiki.mozilla.org/B2G/QA/2014-10-20_Performance_Acceptance#Music

Since there is a large discrepancy in the numbers, it might be worth comparing test parameters, particularly how many songs you're loading and when you start/stop the clock. Those are all documented for what we do in the overview/methodology sections of the studies.
Keywords: perf
Being tracked as part of multi-core performance improvements: https://bugzilla.mozilla.org/show_bug.cgi?id=10935
blocking-b2g: 2.2? → ---
No longer blocks: B2G-Multicore
Depends on: 1107259
No longer blocks: AppStartup
Depends on: AppStartup
Blocks: AppStartup
No longer depends on: AppStartup
Depends on: 835679
feature-b2g: --- → 2.2+
Summary: Music app launch latency is worse compared to Android on quad core 1 GB device → [meta] Music app launch latency is worse compared to Android on quad core 1 GB device
Please remove 2.2+
Flags: needinfo?(bchien)
As discussion with Kevin and Ken, this is in v2.2 feature scope. Marked this for scope tracking.
Flags: needinfo?(bchien)
This should not be, because this bug would over the scope of v2.2. only bugs opened for this bugs should be in the scope.
Flags: in-moztrap?(edchen)
QA Contact: edchen
QA Whiteboard: [2.2-feature-qa+]
Assignee: nobody → bchien
blocking-b2g: --- → 2.2?
feature-b2g: 2.2+ → ---
Any recommendation on the gaia side? Our understanding is that Thinker's team is working on gecko level changes that may improve the app start-up time on quad core. 

Thinker: Do we have launch latency results for music with the gecko optimizations?

Thanks
Hema
Flags: needinfo?(tlee)
Whiteboard: [CR 786374]
Whiteboard: [CR 786374] → [caf priority: p2][CR 786374]
QA team has a report for Nexus 5.
Flags: needinfo?(tlee) → needinfo?(whsu)
Test result. FYI.
- https://docs.google.com/a/mozilla.com/spreadsheets/d/1Hk9kWld3ZdzFxG3vS5au0sepm7eLv2_7BS2Kn3N8Z-A/edit#gid=9028135

Music cold launch time (Visually complete):
@ Workload: 20 songs (make reference-workload-light)
 - Flame 2.1 (Dual-Core & 1024 MB): 1022.1 ms
 - Flame 2.1 (Single-Core & 319 MB): 1968.8 ms
 - Nexus 2.2 (Quad-Core & 2048 MB): 732.8 ms
Flags: needinfo?(whsu)
As all sub cases are resolved, marked as feature-b2g:2.2+
blocking-b2g: 2.2? → ---
feature-b2g: --- → 2.2+
Flags: in-moztrap?(edchen) → in-moztrap-
Coding effort completed. Require verification.
Status: NEW → ASSIGNED
feature-b2g: 2.2+ → ---
Blocks: CAF-v3.0-FL-metabug
No longer blocks: CAF-v3.0-FL-metabug
CAF (Inder) expressed a preference to track performance meta bugs like these instead of their more granular blockers since, in their opinion, the individual blockers tend to change and don't always reflect a direct connection to the primary performance issue. CAF will track and update (close) these bugs when the reported performance issue is resolved.
Cold launch time (Visually Loaded) on Flame:
Datazilla    Master    v2.2    v2.1
Camera       1375      1383    1634
Clock        1266      1164    1041
Contacts      860       806     883
FM Radio      627       554     542
Gallery      1000       940     969
Messages     1392      1255    1262
Music        1312      1164     929 
Phone         615       554     567
Settings     2537      2573    2586
Video        1054      1022     964
From my test the regression between 2.1 and 2.2 is cause by Gecko. Music from gaia 2.2 running on Gecko 2.1 has a similar performance as 2.1
Should we expect that a patch for bug 1138620 will improve startup time for the Music app, or does that patch only help with apps that use the RIL or geolocation?
Flags: needinfo?(mvines)
All apps improve by ~2 seconds.  We've already got a temporary fix for that bug over here though, and I don't think the Mozilla Flame builds were ever affected.
Flags: needinfo?(mvines)
Bobby: can you tell us where the numbers in comment #12 come from?  I can't find them in datazilla myself.
Flags: needinfo?(bchien)
(In reply to David Flanagan [:djf] from comment #16)
> Bobby: can you tell us where the numbers in comment #12 come from?  I can't
> find them in datazilla myself.

The numbers of Comment 12 came from Datazilla[1]. Raptor[2] shows similar numbers as well.

[1] Datazilla: http://goo.gl/86sLQX
[2] Raptor: http://goo.gl/cLR5wc
Flags: needinfo?(bchien)
Thanks. Looks like the 2.2 music number is from 3/31. The corresponding 2.1 number from the same date is 949 in Datazilla, not 929. Still, it is a big regression.

Also, for gallery, comment #12 shows that gallery is faster in 2.2 than in 2.1. But that is not what I measure, and I don't see that 940ms number anywhere in datazilla.
This bug was filed to compare FirefoxOS to Android, but we've spent a lot of time discussing the FirefoxOS v2.1 vs v2.2 performance regression here.  I've now filed bug 1153985 for that specific 2.1 to 2.2 regression.
Depends on: 1153985
No longer blocks: CAF-v2.2-metabug
Last update as followings, and Raptor dashboard [1]. Mark worksforme per no longer 2.2 blocker. Please follow bug 1194650 in v2.5.

-------------------------
appName   master  v2.2
--------  ------  ----
Messages  1700    986
Video     1127    1020
Settings  2682    2437
FM Radio  707     496
Gallery   1076    1046
Music     1085    956
E-Mail    463     1973
Contacts  1118    717
Clock     1202    1206
Calendar  1476    1342
Camera    1668    1297

[1] http://raptor.mozilla.org/#/dashboard/file/raptor.json
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
tracking-b2g: --- → -
Resolution: --- → WORKSFORME
See Also: → https://bugzilla.mozilla.org/show_bug.cgi?id=1194650
You need to log in before you can comment on or make changes to this bug.