See test results at: https://wiki.mozilla.org/B2G/QA/2014-10-02_Performance_Acceptance#Camera Median startup time was 1577 ms for 450 launches. This is also a regression from 2.0, which was median 1485 ms for 390 launches. Graphs, raw data and other testing details are on the wiki.

[Blocking Requested - why for this release]: performance is over acceptance threshold

Requested mtreese to help follow up on the benchmarks from the CAF comparison testing as well. Leaving an NI on him

(In reply to Geo Mealer [:geo] from comment #0) > Median startup time was 1577 ms for 450 launches. This is also a regression > from 2.0, which was median 1485 ms for 390 launches. Camera startup did not regress between 2.0 and 2.1, the positioning of the moz-app-visually-complete performance event changed till *after* the camera's preview stream was confirmed flowing. This added 200-300ms to the datazilla figures. We actually improved startup-time between 2.0 and 2.1, which help to balance some of the perceived regression. I don't think there is much/any margin for optimization left on the Camera app client-side, but I'd love someone to prove me wrong :)

(In reply to Wilson Page [:wilsonpage] from comment #3) > (In reply to Geo Mealer [:geo] from comment #0) > > Median startup time was 1577 ms for 450 launches. This is also a regression > > from 2.0, which was median 1485 ms for 390 launches. > > Camera startup did not regress between 2.0 and 2.1, the positioning of the > moz-app-visually-complete performance event changed till *after* the > camera's preview stream was confirmed flowing. This added 200-300ms to the > datazilla figures. > > We actually improved startup-time between 2.0 and 2.1, which help to balance > some of the perceived regression. > > I don't think there is much/any margin for optimization left on the Camera > app client-side, but I'd love someone to prove me wrong :) Yep, thanks for the clarification. I mentioned this possibility in the dev-gaia post, but couldn't remember for sure which app(s) that happened with so was hoping devs would push back where necessary. Re: optimization, if that's the case (and with the preview stream, that sounds plausible to me) it would be useful to determine what a reasonable minimum launch time is going to be for Camera so we can carry that requirement forward.

One such optimization is in bug 1068393. I'm waiting for a rebased patch from the submitter.

Hema and I have asked MVines for QRD data for comparison purposes - we are waiting for that data.

We won't be able to get 2.1 numbers from QRD testing that matches up close to flame configuration. We will wait for another run of performance tests by Geo on 2.1 codebase. Bug 1068393 will be targeted for 2.2 and there are no other optimizations planned on the app side for 2.1. If folks have any other suggestions for tuning, let us know. Leaving it in the nom queue for another round of results per Tony's request. Thanks Hema

Let's clear the 2.1 blocking nom for now (see comment 3) and re-nom if partner numbers show a regression.

Updated numbers are at https://wiki.mozilla.org/B2G/QA/2014-10-20_Performance_Acceptance, and there's an active thread on dev.gaia concerning them and revising the threshold. https://groups.google.com/d/topic/mozilla.dev.gaia/iakko7aHgi0/discussion

Camera's median startup is tested at 1553 ms as of the 10-31 acceptance tests. https://wiki.mozilla.org/B2G/QA/2014-10-31_Performance_Acceptance#Camera

Main stream is moving to 2.5 on Flame and Aries (Sony Z3C). FxOS 2.1 is low priority now. Mark as wontfix. Please reopen if any special request.