This trivially adds the tests to jit-test, though its unclear if this is the right way to do it. We're not 100 percent sure these are the right tests, but it's what I found in http://hg.ecmascript.org/tests/test262/sputnik_converted. However, it needs some kind of manifest to say "we know these 300 tests fail". This will be much nicer to do after bug 638219 lands, but I could also just hack something trivial together. I need to look more at what the test262 harness does here.

This adds support for the test262 tests. The patch was too big with the tests themselves included, so I'll leave interested parties to get them themselves: - hg clone http://hg.ecmascript.org/tests/test262/ - cp -R test262/test/suite/sputnik_converted tracemonkey/js/src/jit-test/tests/ - cp -R test262/test/suite/ietestcenter tracemonkey/js/src/jit-test/tests/ This patch is a hack, but it does roughly what it's supposed to. (I'll make it work nicely and try and land it once bug 638219 lands). Roughly 5000 tests fail, largely because the ietest are nearly all UTF8+BOM, which we do a bad job with. Other than that, we appear to have the right number of failures (ie we take into account broken tests and the tests which test error conditions and end up with 200ish failures from sputnik).

With this fix (and the BOM), we've got a total of 370 fails, which I think is the same as Waldo found. This needs a manifest to list the tests we know we fail at. After that, I'm not sure the best way to include it, as it adds 10 minutes to the test suite run-time.

Minor update: when we have failures, handle and report them properly.

Some tests have been disabled recently, so this takes it into account.