I am working on getting Android Sync's JUnit 3 instrumentation tests ready to run on buildbot. See Bug 709353 for the (mostly complete) dependency tree. I am finding that adding my fresh APK to UPLOAD_EXTRA_FILES in packager.mk is leading to bizarre testing errors on buildbot. I've looked into mozharness but I can't make progress, since it looks like the relevant installer_url is wrong. Two try builds that show the issue: broken: https://tbpl.mozilla.org/?tree=Try&rev=132f0d9985c5 working: (without UPLOAD_EXTRA_FILES): https://tbpl.mozilla.org/?tree=Try&rev=0014a4798289 Can someone dig into this?

Which error exactly is confusing? I took a brief look and all I see are known intermittent errors

seems like you broke the expected file names. Android 2.2 (tegra) doesn't run mozharness and won't. I see this (https://tbpl.mozilla.org/php/getParsedLog.php?id=28186409&tree=Try&full=1#error0): python mochitest/runtestsremote.py --deviceIP 10.250.51.193 --xre-path ../hostutils/xre --utility-path ../hostutils/bin --certificate-path certs --app 'cat: package-name.txt: No such file or directory' --console-level INFO --http-port 30353 --ssl-port 31353 --pidfile /builds/tegra-353/test/../runtestsremote.pid --run-only-tests android.json --symbols-path=http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/nalexander@mozilla.com-132f0d9985c5/try-android/background.crashreporter-symbols.zip --total-chunks 8 --this-chunk 1 notice the --app is set to 'cat: package-name.txt: No such file or directory'. I suspect you broke the files uploading. I also see: ========= Started Install App on Device (results: 0, elapsed: 2 mins, 3 secs) (at 2013-09-20 20:07:46.287226) ========= python /builds/sut_tools/installApp.py 10.250.51.193 build/background.apk 'cat: package-name.txt: No such file or directory' I have no idea what build/background.apk is, I suspect we install the first .apk found in a directory listing.

(In reply to Ben Hearsum [:bhearsum] from comment #1) > Which error exactly is confusing? I took a brief look and all I see are > known intermittent errors For example, from https://tbpl.mozilla.org/php/getParsedLog.php?id=28186426&tree=Try&full=1 It's trying to download background.apk (instead of fennec-*?): ========= Started download (results: 0, elapsed: 0 secs) (at 2013-09-20 20:07:05.057530) ========= wget --progress=dot:mega --tries=5 --waitretry=120 -N http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/nalexander@mozilla.com-132f0d9985c5/try-android/background.apk It then starts complaining that it can't extract package-name.txt: remoteProcessName: 'cat: package-name.txt: No such file or directory'

(In reply to Joel Maher (:jmaher) from comment #2) > seems like you broke the expected file names. Android 2.2 (tegra) doesn't > run mozharness and won't. I see this > (https://tbpl.mozilla.org/php/getParsedLog. > php?id=28186409&tree=Try&full=1#error0): > python mochitest/runtestsremote.py --deviceIP 10.250.51.193 --xre-path > ../hostutils/xre --utility-path ../hostutils/bin --certificate-path certs > --app 'cat: package-name.txt: No such file or directory' --console-level > INFO --http-port 30353 --ssl-port 31353 --pidfile > /builds/tegra-353/test/../runtestsremote.pid --run-only-tests android.json > --symbols-path=http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/ > nalexander@mozilla.com-132f0d9985c5/try-android/background.crashreporter- > symbols.zip --total-chunks 8 --this-chunk 1 > > > notice the --app is set to 'cat: package-name.txt: No such file or > directory'. I suspect you broke the files uploading. > > I also see: > ========= Started Install App on Device (results: 0, elapsed: 2 mins, 3 > secs) (at 2013-09-20 20:07:46.287226) ========= > python /builds/sut_tools/installApp.py 10.250.51.193 build/background.apk > 'cat: package-name.txt: No such file or directory' > > > I have no idea what build/background.apk is, I suspect we install the first > .apk found in a directory listing. That's the new APK I am trying to upload. My guess is we're installing the first .apk too; but it doesn't seem to matter if the new APK is background.apk or instrumentation-tests.apk, so we're not sorting alphabetically. So "installing the first APK" is fragile(!); how do we fix that? And where is the code that actually makes this go?

maybe it is the order of the files in the variable. Is it possible you have messed up the file list?

(In reply to Joel Maher (:jmaher) from comment #5) > maybe it is the order of the files in the variable. Is it possible you have > messed up the file list? It is technically possible; but UPLOAD_EXTRA_FILES is strictly a build-time thing, right? And this is failing during testing. I will push a try build with my APK *after* robocop.apk just in case, but this seems unlikely. And if this is the case, we should address it so the buildbots don't have a secret ordering requirement.

> I will push a try build with my APK *after* robocop.apk just in case, but > this seems unlikely. Here's a try with background.apk added to UPLOAD_EXTRA_FILES after robocop.apk: http://tbpl.mozilla.org/?tree=Try&rev=5bd93c83739f

Updated try, since the logs have been purged http://tbpl.mozilla.org/?tree=Try&rev=1916c601099e I am going to need movement on this soon.

(In reply to Nick Alexander :nalexander from comment #4) > So "installing the first APK" is fragile(!); how do we fix that? And where > is the code that actually makes this go? http://hg.mozilla.org/build/buildbotcustom/file/f5f327ef0ded/process/factory.py#l203 The problem is, how do we tell if it's the installer or not? We don't hardcode fennec/firefox since you can rename the installer, iirc, plus that's not great for thunderbird/seamonkey/etc.