https://hg.mozilla.org/integration/mozilla-inbound/rev/8526c7a38761 from bug 951354 created TestCertDB. It failed on Win8 debug and 10.6/10.8 debug Cpp runs. It was backed out in https://hg.mozilla.org/integration/mozilla-inbound/rev/41a0bf080410. Then it kept failing after the backout, on test runs against builds created on slaves that had built while it was in (and if you look at tbpl through Friday night/Saturday, you'll see foolish me creating a situation that deeply confused me, by setting a clobber, then retriggering on the pushes while it was in, thus creating even more slaves which would need to be clobbered to notice the backout).

my theory, without looking at the code or the logs, is that we're not creating test zips, we're updating existing ones. So while dist/bin doesn't have the test after the backout, the zip still has.

cd dist/test-package-stage && \ zip -rq9D '/builds/slave/m-in-osx64-d-00000000000000000/build/obj-firefox/dist/firefox-29.0a1.en-US.mac64.tests.zip' \ * -x \*/.mkdir.done so yeah, unless we rm dist/ at some point earlier, we would be. Do we want to add -FS to that, so we can actually remove cpp and jit tests?

We do have: http://mxr.mozilla.org/mozilla-central/source/testing/testsuite-targets.mk#433 Which rm's the package stage dir before staging. I suspect the problem is actually here: http://mxr.mozilla.org/mozilla-central/source/testing/testsuite-targets.mk#489 We copy the CPP_UNIT_TEST binaries to dist/cppunittests during the build: http://mxr.mozilla.org/mozilla-central/source/config/rules.mk#123 but there's no facility for removing things from that directory. This probably wants to get hooked into an install manifest from moz.build instead.

This is likely still a bug. testsuite-target.mk still does horrible things for cppunittests. Please leave comments about clobber issues when you encounter them so we can prioritize accordingly.