Currently: try: -b d -p android-api-15 -u mochitest-gl[Android 4.3 API15+] -t none Should be: try: -b d -p android-api-15 -u mochitest-gl[Android 4.3] -t none Otherwise the tests will no trigger.

Good times, when it was that simple. Now, debug Android tests are run by taskcluster, and although it was willing to match the platform exclusions for "Ubuntu" and "x64", it has ignored all the rest, insisting instead on its own test_platform, undocumented other than as the part before a / in the keys in a test_platforms.yml file (or, if you want to really apply a whole lot of outside knowledge, as the part after (desktop|android|b2g)-(build|test)- and before a / in the "Job name" as displayed on Treeherder, which may or may not always also be the "Build" minus leading and trailing dashes as displayed on Treeherder for a test, dunno). So to get your example to run, you would need to use "try: -b d -p android-api-15 -u mochitest-gl[android-4.3-arm7-api-15] -t none" and to get opt and debug you would need to use "try: -b do -p android-api-15 -u mochitest-gl[Android 4.3,android-4.3-arm7-api-15] -t none".

My first step in touching trychooser is always to get it as close to validating as possible, since doing so almost always fixes a bug in it. In this case, not a very interesting bug: if it actually worked to have two sets of three elements each all with the same id, then if we actually had a source for build and test load we would have been claiming that the Android Gradle builds and Android frontend builds which run on taskcluster and have no tests each had the load of the builds and tests on buildbot. The <label> was just a stray.

The last time Android 2.3 existed was in late June on mozilla-release.

In theory, what would have been "right" would be to use "Android 4.3 armv7 API 15+" since |"Android 4.3 armv7 API 15+" in "Android 4.3 armv7 API 15+ mozilla-inbound opt test mochitest-gl-1"| is true, while |"Android 4.3 API 15+" in "Android 4.3 armv7 API 15+ mozilla-inbound opt test mochitest-gl-1"| is not. However, that doesn't work for some reason, and anyway our goal with these is not to match the entire prettyname string, it's to use as short a string as possible to distinguish a platform from any other platform. Since we're only running one set of Android 4.3 tests, that's enough.

Comment on attachment 8792359 [details] [diff] [review] part 1, the actual fix Lgtm.

Comment on attachment 8792358 [details] [diff] [review] part 0.2, there is no 2.3 We stopped building and testing Android API 9-10 in Firefox 48 (bug 1250994), so dropping the corresponding entries from TryChooser seems the right thing to do here.

Comment on attachment 8792357 [details] [diff] [review] part 0.1, a little validity I'm not really familiar with the behaviour of Android Gradle and Frontend jobs, so I'll defer this review to Kim.

https://hg.mozilla.org/build/tools/rev/a986f8e1dddf https://hg.mozilla.org/build/tools/rev/f67d5d1c2315 https://hg.mozilla.org/build/tools/rev/fc88dd15963b

Updated TryChooser to account for the changes above.