The existing raptor reboot test (boot_test.js) doesn't work on the b2g-emulator, because 'adb reboot' is not supported on the emulator. It is not possible to do a simulation of a hard reboot on the emulator. I was considering on emulator doing a soft-reset (b2g restart) instead of a reboot. Not quite the same but I believe measuring the b2g process restart time would still provide significant value for an FXOS perf test (and just using it to establish a baseline to detect regressions; not as actual real reboot performance numbers). Eli, I was thinking I would add a new method in /lib/device/index.js called "rebootEmulator", which would do the same as "reboot" with regards to markers etc. however insead of an 'adb reboot' it would just do a b2g restart (adb stop b2g && adb start b2g) and would do a 'waitForHomescreen" (instead of "adb wait-for-device since when just restarting b2g the emulator is always running). Then in the /lib/suite/reboot.js itself, if options.emulator is set then invoke 'rebootEmulator' instead. What do you think? Or should I create a /lib/emulator/index.js (although lots will be duplicate)? Or should I create a new perf test called 'reset_test', which would work on both device and emulator, and then we just won't run the existing 'boot_test' on the emulator? Thanks in advance for your thoughts!

I think adding a new suite phase for only restarting B2G is in order. It would function similar to the Reboot suite, but rather than doing the device reboot it would do the B2G restart. Most of the code for this to happen already exists spread through Reboot and ColdLaunch, so it may also be in order to push some of that into the base Phase for re-use. Then we can create a new test separate from reboot_test (e.g. b2g_restart_test) which would consume the new phase and be run by the simulator. Does that sound reasonable?

Yes that sounds great. I will put a patch together for this, but will wait until the MozDevice work (Bug 1115134) has landed.

Add restart-b2g phase to raptor.

Add restart-b2g raptor test.

Comment on attachment 8559811 [details] https://github.com/mozilla-b2g/raptor/pull/13 My main concern is just getting the time from the device rather than overriding with the time from the Node process. Mozdevice will need to be modified to fetch the device time during the B2G restart process. Other than that, just a couple nits to address.

Comment on attachment 8559811 [details] https://github.com/mozilla-b2g/raptor/pull/13 Thanks for the review Eli. PR updated, I'll squash the commits after review.

Corresponding change to mozdevice, to return device time before b2g restart.

Comment on attachment 8560539 [details] https://github.com/eliperelman/node-mozdevice/pull/5 Before merging this, could you also include a bump to the package.json to 1.2.2?

(In reply to :Eli Perelman from comment #9) > Comment on attachment 8560539 [details] > https://github.com/eliperelman/node-mozdevice/pull/5 > > Before merging this, could you also include a bump to the package.json to > 1.2.2? Right, I had that change locally but forgot to add it to my commit, d'oh, anyway thanks, done & merged: https://github.com/eliperelman/node-mozdevice/commit/7eb661b20d6b14ccc71e939df02c82360eb0225a

Comment on attachment 8559811 [details] https://github.com/mozilla-b2g/raptor/pull/13 PR updated as per our conversation on IRC (and your github gist for re-organizing phase.format).

Comment on attachment 8559811 [details] https://github.com/mozilla-b2g/raptor/pull/13 Awesome job on this. R+ with nits fixed.

Thanks Eli. Rebased, nits fixed, squashed and merged: https://github.com/mozilla-b2g/raptor/commit/ec63a747ae6d272caa4c174c143b25cbd8143a6b

Update gaia-node-modules.

Comment on attachment 8561454 [details] https://github.com/mozilla-b2g/gaia-node-modules/pull/91 Sounds good. Feel free to update the gaia revision as well.

Thanks Kevin. Merged: https://github.com/mozilla-b2g/gaia-node-modules/commit/25756ef8ae9cb1e9a0c9bc893612bb8334af3ccf

(In reply to Autolander from comment #18) > Created attachment 8561590 [details] > [PullReq] rwood-moz:bug1121187 to mozilla-b2g:master Green try run. Carrying fw the R+ from Eli, and have R+ from Kevin. Merged: https://github.com/mozilla-b2g/gaia/commit/0dd0a8c8d96969398e9e04da11327d54d2569935