+++ This bug was initially created as a clone of Bug #638219 +++ We should move these tests out of |make check| and into a new test "job, as we can "move them to run on the test machines from the test package, since we could also then run them on Android devices fairly easily." Credits go out to Ted and Gregory for the original description in bug 638219.

I think we might want to keep them in "make check" for local builds, since that's what JS developers are used to. We can add a mozconfig option like "NO_JIT_TESTS_IN_CHECK=1" in the in-tree mozconfigs to make them skip the tests in check once we have them running from packaged builds.

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #1) > I think we might want to keep them in "make check" for local builds, since > that's what JS developers are used to. We can add a mozconfig option like > "NO_JIT_TESTS_IN_CHECK=1" in the in-tree mozconfigs to make them skip the > tests in check once we have them running from packaged builds. I'm pretty sure no JS developer has ever run 'make check' unless it already failed on tinderbox :-). We just run jit-tests manually.

Coop: Is running JIT tests from a standalone package (not as part of make check) on your radar? If not, here you are! Currently, |make check| runs as part of every build job. This includes build-only try pushes. On Windows, |make check| appears to take ~55 minutes on current RelEng hardware, or roughly about the length of compilation. If we actively move things out of |make check| and into standalone test jobs, this would go a long way to reduce the load on the builders. This should hopefully reduce overall infra load as try builds not wanting |make check| can omit it. If we ever go the route of selectively running jobs based on files that changed, having extra knobs for the pieces in |make check| may also be useful. The JS JIT tests currently consume a large portion of |make check|'s execution time and should be relatively easy to extract into a standalone job. So, I think it makes sense to prioritize moving them out of |make check|. I guess next steps are packaging the JIT tests so they can be executed standalone then working with RelEng to configure a new builder for them.

Bug 858620 - Add jit-tests to test package Bug 858621 - Run jit-tests from test package also: Bug 858622 - Make jit-tests runnable on mobile (is the nice side benefit of this)

(In reply to Gregory Szorc [:gps] from comment #3) > Coop: Is running JIT tests from a standalone package (not as part of make > check) on your radar? If not, here you are! It's on my radar now. > I guess next steps are packaging the JIT tests so they can be executed > standalone then working with RelEng to configure a new builder for them. Happy to find an owner for the releng followup work that will come out of the bugs Ted filed (comment #4). Is there any immediate releng action WRT |make check|, or are we just planning to remove the JIT tests from |make check| once the JIT tests are running standalone?

Per comment 2, JS developers don't actually care about having them in make check.