When trying to run browser-chrome mochitests written for fennec, we don't find them in the package-tests file. This looks to be the case because all of the unittests live in the xulrunner objdir, and our mobile specific tests live int he mobile objdir. I tried a make package-tests in the mobile objdir and received errors in the script while it was trying to cd ../../dist/bin/plugins. My recommendation is to create a cleaner make package-tests that can be run from the mobile objdir. If we go this route, we will need releng to create another tests binary for us for each build in addition to the normal tests binary. Ted, you might have ideas here.

I'm surprised that we didn't already have a bug on this, but yeah, as currently implemented "package-tests" can't know about the Fennec objdir, because it's completely separate from the xulrunner objdir. It's the same situation as "make package", really, where the mobile tree jumps through some hoops to package the xulrunner bits. (Also, I think we should just change Fennec builds to not be a two-pass build, but that belongs in a separate bug.)

patch to package tests for mobile similar to 'make package-tests' but in the mobile tree. Run it by doing: cd $(objdir)/mobile/mobile/installer make package-mobile-tests cd $(objdir)/mobile/dist ls *tests*.bz2 One caveat: if you run this first, then $(objdir)/mobile/make package, make package will delete the $(objdir)/mobile/dist folder and your bz2 file will disappear. Another thing to note is this will turn the tests into the same testing structure as the xulrunner (firefox) *tests.tar.bz2 file. So if you untar this in the same directory, it will add the additional mobile browser-chrome tests to mochitest/browser/mobile/*. I testing this with a fresh clobber build on mozilla-central and 1.9.2 for desktop linux Fennec. I ran the tests in a virgin directory (make package + make package-mobile-tests only- untar'd to a fresh dir) like so: cd mochitest python runtests.py --appname=../fennec/fennec --xre-path=../fennec/xulrunner --browser-chrome --autorun

Comment on attachment 407663 [details] [diff] [review] package mobile tests Seems ok to me, but Ted should really have a look

It would be nice if we could run the make package-mobile-tests in the objdir too. This certainly gets us moving in the right direction: running browser-chrome tests!

Comment on attachment 407663 [details] [diff] [review] package mobile tests Bleh. Just include $(topsrcdir)/testing/testsuite-targets.mk, and do package-mobile-tests: stage-mochitest Then tar up the result? Ideally you could just reuse the package-tests target, but we'd have to do some trickery in there to not try to package all the reftest stuff, etc.

I would like to have it reuse the same logic and code if possible. I found that what we do for mochitest stage-package (http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/Makefile.in#127), has issues with the /bin and /certs stuff since dist is $(objdir)/mobile vs $(objdir)/xulrunner.

Yeah. I'd take a patch that adds ifdefs or handles failures there silently to make it work.

patch to allow $(objdir)/mobile/mobile: make package-mobile-tests reuse the same logic for package-tests as xulrunner.

this is the other part of the patch that uses #ifdef's to fix the 'make package-mobile-tests' from the mobile objdir.

Comment on attachment 408537 [details] [diff] [review] package-tests for mozilla-central/1.9.2 +#ifdef WINCE +DIST_BIN = $(DIST)/bin/xulrunner +#endif + +#ifdef MOZ_PLATFORM_HILDON +DIST_BIN = $(DIST)/bin/xulrunner +#endif This is no good. For one, we already have Firefox WinCE builds, and we'll want to run tests on those. Also, when do the xulrunner bits get stuck in the Fennec dist/bin? You'll want to make sure there isn't an ordering problem between running "make package" and "make package-tests". I think if you just did DIST_BIN = $(DIST)/bin, then you could just have the mobile Makefile call stage-mochitest directly, like package-mobile-tests: $(MAKE) stage-mochitest DIST_BIN=$(DIST)/bin/xulrunner and that would be ok.

Comment on attachment 408536 [details] [diff] [review] package-tests for the mobile-browser branch This looks ok, although per my other review comment you might want to change the way this calls stage-mochitest.

update for package-mobile-tests for the mobile-browser branch from last review comments.

updated patch for package-mobile-tests on mozilla-central and 1.9.2. Much simpler and tested from fresh clobber builds on both branches.

just verified with a fresh m-c/m-b sync up and clean clobber build that this has no bitrot for patches or functionality. And the mobile browser-chrome tests still pass! mfinkle, please check this in when you get a chance.

pushed m-b: https://hg.mozilla.org/mobile-browser/rev/0a42c675f8a7 pushed m-c: http://hg.mozilla.org/mozilla-central/rev/15726bd6943c pushed m-192: http://hg.mozilla.org/releases/mozilla-1.9.2/rev/2f5c1da8fe1c