James Lal has implemented a Python testrunner for the Gaia Unit tests, at https://github.com/lightsofapollo/js-test-agent/tree/master/python/test_agent. We can adapt this testrunner to work better with buildbot: - the testrunner should take care of launching the b2g desktop build and stopping it at the end of the test - the testrunner should report status in a way that's compatible with TBPL - (potentially) we might want to use some lighter-weight packages for the WebSocket server, instead of twisted/zope.

Another goal of this rewrite is to add a substantial amount of logging, so that when jobs fail in buildbot, we'll have some idea why.

I've put lightsofapollo's Python testrunner in a new repo at https://github.com/jonallengriffin/gaia-unit-tests so I can work on it; when it's ready I imagine we'll move it into the Gaia repo.

@mdas, currently you can invoke the testrunner with this command-line: python main.py --binary /path/to/b2g/binary --profile /path/to/gaia/profile calendar/test/unit/calc_test.js for example. Test files are always specified relative to the 'apps' directory in Gaia. At present, until bug 840268 is fixed or I add a Marionette dependency, you'll need to manually launch the "Test Agent" app after the build is launched.

I've updated the test runner so that it will run all the tests when launched without a test name, so you can just do this: python main.py --binary /path/to/b2g/binary --profile /path/to/gaia/profile The runner will automatically launch the Test Agent app now that bug 840268 is fixed, although this currently only works on gaia-master and mozilla-central.

Landed in gaia: https://github.com/mozilla-b2g/gaia/commit/c979d39c555f523213affc324609a86b83303491