We need to establish if there are any issues when running the tests against KitKat builds. For this we should perform a flash using the v180 base build and then full flash the appropriate build on top. We should test all branches currently in use in automation using as close to the same configuration (for example, memory allocation).

I have run performance tests using b2gperf against the tinderbox builds for b2g-inbound, mozilla-aurora, and mozilla-b2g32_v2_0 with no unexpected issues. Once nightly builds are available for these I will repeat the testing using the new builds.

I ran the smoke test on B2G-inbound on a flame with v180 + shallow flash b2g-Inbound Flash command: ./flash_tbpl.py -d flame-kk -v b2g-inbound --eng -G -g Build info: Gaia bc48cbd9e7b871cc648a525761bd6d512e91d741 Gecko https://hg.mozilla.org/integration/b2g-inbound/rev/368d56884b3a BuildID 20140916002244 Version 35.0a1 ro.build.date Thu Sep 4 14:59:02 CST 2014 ro.bootloader L1TC10011800 ro.build.version.incremental 27 Tests run as expected without any major issues

Bebe: Thanks, can you also run with full flash as mentioned in comment 0? Also, please test the tinderbox builds for mozilla-aurora and mozilla-b2g32_v2_0.

Yes Dave Running the tests now for both shallow and full flash I'm adding the results as test runs finish

Ran tests on v180 + full flash b2g-Inbound Flash command: ./flash_tbpl.py -d flame-kk -v b2g-inbound --eng -f Build info: Gaia 25616a8cf9ab2cc98cb5e4d5323a3537a96d14a2 Gecko https://hg.mozilla.org/integration/b2g-inbound/rev/693b8e8605e2 BuildID 20140916013747 Version 35.0a1 ro.build.date Tue Sep 16 04:54:22 EDT 2014 ro.bootloader L1TC10011800 ro.build.version.incremental eng.cltbld.20140916.045411 Tests run as expected without any major issues

Ran tests on v180 + full flash mozilla-central Flash command: ./flash_tbpl.py -d flame-kk -v mozilla-central --eng -f Build info: Gaia e2d70bee03b5380ac327a145e5d694fb2443f018 Gecko https://hg.mozilla.org/mozilla-central/rev/0e581e529fd6 BuildID 20140915201445 Version 35.0a1 ro.build.date Mon Sep 15 23:24:28 EDT 2014 ro.bootloader L1TC10011800 ro.build.version.incremental eng.cltbld.20140915.232418 Tests run as expected green build

Ran tests on v180 + shallow flash mozilla-central Flash command: ./flash_tbpl.py -d flame-kk -v mozilla-central --eng -G -g Build info: Device firmware (date) 04 Sep 2014 09:59:02 Device firmware (incremental) 27 Device firmware (release) 4.4.2 Device identifier flame Gaia date 15 Sep 2014 20:39:43 Gaia revision e2d70bee03b5 Gecko build 20140915201445 Gecko revision 0e581e529fd6 Gecko version 35.0a1 Tests run as expected there was a crash but not related with the current bug

Ran tests on v180 + full flash mozilla-aurora Flash command: ./flash_tbpl.py -d flame-kk -v mozilla-aurora --eng -f Build info: Gaia 5decbb091f09977aa7cdddd8bc4eca0a9b5e9fbe Gecko https://hg.mozilla.org/releases/mozilla-aurora/rev/932702704ff7 BuildID 20140916024235 Version 34.0a2 ro.build.date Tue Sep 16 07:31:27 EDT 2014 ro.bootloader L1TC10011800 ro.build.version.incremental eng.cltbld.20140916.073116 Tests run as expected without any major issues

Ran tests on v180 + full flash mozilla-b2g32_v2_0 Flash command: ./flash_tbpl.py -d flame-kk -v mozilla-b2g32_v2_0 --eng -f Build info: Gaia 7edd3b0b9f65c3dde235c732d270e43e055a1254 Gecko https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/0ef9688da1b6 BuildID 20140916051646 Version 32.0 ro.build.date Tue Sep 16 08:35:56 EDT 2014 ro.bootloader L1TC10011800 ro.build.version.incremental eng.cltbld.20140916.083545 Tests run as expected without any major issues

Thanks Bebe, this gives us great confidence for b2g-i builds. We'll begin to run b2g-i tests on KK. We'll need to re-test again with m-c in the future, thus leaving this bug open.

All b2g-i jobs on Jenkins have now been migrated to use KK base and KK gecko/gaia. Now I'm just managing the ratio of nodes for KK/JB jobs.

All tests are OK on the nightly builds.