Currently you have to build Mozilla to be able to run Mochitest. This puts up an artificially high barrier to anyone wanting to write new Mochitests. If it was possible to run Mochitest with nightly builds, we'd enable/encourage a far larger group of people to contribute since they'd then be able to run the tests they write.

Related: http://ted.mielczarek.org/code/mozilla/mochitest-maker/ That at least lets you write simple Mochitests that don't need chrome access or special HTTP things. (Which probably covers a large portion of Mochitests.)

Or, a <a href="http://xoatlicue.blogspot.com/2008/01/standalone-test-product.html">standalone product</a> can be provided that contains the tests and which can be pointed at any executable.

And I found this also. create a downloadable test executor image https://bugzilla.mozilla.org/show_bug.cgi?id=378090

I have actually thought about this and tried a bunch of things. I was hopeful that one could be able to define a test project that one could build in the standard "mozilla-ish" way. In other words, check out a client.mk, make checkout MOZ_CO_PROJECT=tester, ./configure --enable-application=tester, make. Like that. I do not think this will be possible, or very easy. There is something that can be done that is pretty easy. Build Firefox with tests and debug and everything turned on. Then, in a script, put the testing executables into their own "product". One of the good things about this is that a single "tester" image can be used with many different FF builds. Of course, between two versions very far apart, there will be differences that would cause errors. But between any two reasonable nightly builds, there would be no reason to re-download the tests. One can use such a runner to execute mochitests, reftests and xpcshell tests. In all of these, pieces of the mozilla architecture are used, but the pieces being used to run the tool do not need to be the same version being tested. I even have scripts that add reftest-testability to an app bundle, and that runs a set of reftests. A current version is at http://www.wykiwyk.com/mozilla/tests/tests_20080429/. If you are on a Mac OS X, you may be able to wget this and run it. This directory has a log of being run on 4/29 and I ran it today also. FYI, right now, I can only run with apps I build, but I can point to any number of apps I build. The nightlies are missing something and I have not taken the time to see what. Are you guys looking to going another way with this, or are you amenable to a patch?

I was thinking something similar in bug 422398. I thought I had another bug for packaging the testing bits, but perhaps I don't. It sounds like a pretty sane solution, just package the test bits in a way that could be used separately. I don't think it'd be too hard to add a build target that would just get you the test bits once that was implemented.