https://tbpl.mozilla.org/php/getParsedLog.php?id=24355575&tree=Cedar I can reproduce this locally, so I'll take a stab at figuring out what's happening.

So I believe that the test-container app isn't getting packaged with the new emulator build. If I download "gaia.zip" from the cedar build directory and navigate to "profile/webapps", there is no test-container app there. With the emulator built by jenkins, I can navigate to "b2g-distro/out/target/product/generic/data/local/webapps" and test-container is listed. Catlee, can we package the test-container app (https://github.com/mozilla-b2g/gaia/tree/master/test_apps/test-container) alongside the normal apps? In fact I'd imagine we might want all the test_apps (https://github.com/mozilla-b2g/gaia/tree/master/test_apps)

Morphing bug to new component

Should this be included inside gaia.zip? Or you want separate zips for test_apps?

I think inside gaia.zip would be fine. For the record, I don't really know what the other test_apps are for. I just assume it would be a good idea to include them, but maybe that assumption is wrong. Would I need to take any other steps to get them inside the emulator once they are packaged in gaia.zip (like manually copying the profile over)? Or would it get picked up automatically somehow?

I'm not really sure. Right now we create gaia.zip with this command: zip -r -9 /path/to/gaia.zip gaia/proflie so I should add gaia/webapps to that list? or where would these files be on the build system?

Oh I see. I guess what we want is to make sure when we build gaia profile, the test_apps are included. For example if I run this locally: $ cd gaia $ rm -rf profile $ make profile $ ls profile/webapps all the test_apps are included. Unfortunately I have no idea what is different between my configuration and the official build's that would be different :/

For reference search for "test -d profile || mkdir -p profile" in the log (https://tbpl.mozilla.org/php/getParsedLog.php?id=24406201&tree=Cedar&full=1) to get to the start of the "make profile" step.

So, what we want is for the emulator's built-in gaia profile to be built as an engineering build, not a user build. Are we somehow triggering a user build right now? The contents of gaia.zip are not directly relevant, since we don't actually use that file in the test harness, although they do give a clue as to what's wrong.

This is the result of PRODUCTION=1 in the env during the build. We don't want this set for emulator builds.

We added PRODUCTION=1 in https://bugzilla.mozilla.org/show_bug.cgi?id=855673#c3 through https://bugzilla.mozilla.org/show_bug.cgi?id=855673#c5 . We can: a) revert this line, which may cause the various people who requested PRODUCTION=1 in the first place to wonder why they're seeing test apps. (Or we can discuss beforehand.) It does seem like we have a discrepancy in what we expect releng-produced builds to look like. b) create a releng-emulator.py that is a copy of releng.py, but removes the PRODUCTION=1 flag c) create a commandline option to set PRODUCTION=1. Enable it on all releng.py builders, but don't enable on emulator builds.

We never want PRODUCTION=1 for emulator builds, but they very likely do want these for non-engineering device builds. So, we probably want solution b) or c)

hg cp releng.py releng-emulator.py # remove PRODUCTION=1 from releng-emulator.py

Comment on attachment 767443 [details] [diff] [review] releng-emulator.py http://hg.mozilla.org/build/mozharness/rev/e961d2fa0660

Comment on attachment 767444 [details] [diff] [review] (configs) point at it http://hg.mozilla.org/build/buildbot-configs/rev/cf82f68a6b83 Should be fixed next merge+reconfig (tomorrow?)

in production

We're using the b2g-emulator.py config file, which doesn't have PRODUCTION=1. -> RESO FIXED.