+++ This bug was initially created as a clone of Bug #1224861 +++ We need to improve manifest checking within travis. I suggest we should at least add trying to read all manifests. That would have triggered the bell for avoiding bug 1224861. I don't know if we can go further and check all branches are good for all manifests? We already do that for some devices I guess.

Currently, only b2g_bumber gets checked against a couple of devices: https://hg.mozilla.org/build/mozharness/file/tip/configs/b2g_bumper/master.py I am not fluent enough in that mozharness/bumper bot to know if this might be enough to satisfy checking: would a broken base xml break that? None references the sony-aosp-l manifest. The nexus 4 kk and 5 l are there though in https://hg.mozilla.org/build/mozharness/file/tip/configs/b2g_bumper/v2.5.py and we got no warning so I guess this is not enough.

Successfull travis: https://travis-ci.org/mozilla-b2g/b2g-manifest/builds/91406253 Reproducing bug 1224861: https://travis-ci.org/mozilla-b2g/b2g-manifest/builds/91406638 So this is good :)

Comment on attachment 8687925 [details] B2G Manifest PR I suspect this won't work on merges, since $TRAVIS_PULL_REQUEST won't be set. I think the only way to test would be to set this up on a separate fork in github.

I'm not sure we can rely on Travis verification to avoid bug 1224861, because according to https://github.com/mozilla-b2g/b2g-manifest/commit/99881044aa5a29ac6a6e9f67d4f962ce28f49630 there was not PR involved to merge that.

(In reply to Alexandre LISSY :gerard-majax from comment #6) > I'm not sure we can rely on Travis verification to avoid bug 1224861, > because according to > https://github.com/mozilla-b2g/b2g-manifest/commit/ > 99881044aa5a29ac6a6e9f67d4f962ce28f49630 there was not PR involved to merge > that. Said otherwise, if we do not run Travis when branching, then we should only care about supporting the PRs, and the changes I made should be enough to limit running that against PR.

Any other merge happening should have been done from a reviewed PR, so I guess that would be good enough to catch most of the mistakes.

Should we uplift that to 2.5 ?

Uplifting to v2.5 exposes breakage because of missing base-caf-jb.xml: https://travis-ci.org/mozilla-b2g/b2g-manifest/builds/91652163

(In reply to Alexandre LISSY :gerard-majax from comment #11) > Uplifting to v2.5 exposes breakage because of missing base-caf-jb.xml: > https://travis-ci.org/mozilla-b2g/b2g-manifest/builds/91652163 I have a green build with restoring base-caf-jb.xml: https://travis-ci.org/mozilla-b2g/b2g-manifest/builds/91656235

So, Chris, it's up to you to decide if we uplift all of that. Fixing v2.5 flame (JB based) can be helpful for contributors that are working on devices stuck on JB (Open C for now for example).