Random idea: package-tests is currently a large part of the build time, while also not being parallelized.

OTOH, most of the files that are packaged, and the artifacts produced, don't need to wait for a full build, and even better, are platform-independent. For those that are, it would make sense to move all the work to a separate task, which could probably even be unique for all platforms.

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

We should decide if we're going to do this, or have source checkouts on the test machines. If we had checkouts of gecko on the test machines, then we wouldn't need to package up these test files as tool chains or as part of the build.

can we have a shared storage volume that we attach to the containers- this would reduce the need for bulk clone/updating.

(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #3)

can we have a shared storage volume that we attach to the containers- this would reduce the need for bulk clone/updating.

That's what tasks do already (when things are set up correctly).

For the record, we can kinda trivially generate the following archives from a separate --disable-compile-environment build:

• target.talos.tests.tar.gz
• target.raptor.tests.tar.gz
• target.jsreftest.tests.tar.gz
• target.condprof.tests.tar.gz
• target.awsy.tests.tar.gz

The following can also, assuming that cc_type in mozinfo.json doesn't have an impact:

• target.web-platform.tests.tar.gz
• target.reftest.tests.tar.gz

target.mochitest.tests.tar.gz is almost identical too, except for mozinfo.json as above, but also with --disable-compile-environment is doesn't contain the updater binary, which... I don't know why it's there in the first place... I mean, it should already be part of Firefox.

The above is the result from comparing both artifacts with and without --disable-compile-environment on a single platform, though. That doesn't tell about platform-independence.

Between a linux --disable-compile-environment and windows normal build, the same artifacts are identical (target.talos.tests.tar.gz, target.raptor.tests.tar.gz, target.jsreftest.tests.tar.gz, target.condprof.tests.tar.gz, target.awsy.tests.tar.gz), but there are more differences in the others:

• the mozinfo.json differences are more extensive. It feels like mozinfo.json could be an artifact of its own instead of being in (some) of the archives.
• target.reftests.tar.gz has differences in reftest.jsm because of a #ifdef MOZ_ENABLE_SKIA_PDF in the file (there's also a #ifdef XP_MACOSX, BTW, so that would be more differences yet on mac)
• target.mochitest.tests.tar.gz has more files on Linux.

All these three above are problems that will surface if moving to source checkouts instead.

could we just upload packages from a single build and reuse them for all the other builds? Sounds like the suggestions for mozinfo.json and other small changes would allow that to happen.

(In reply to Mike Hommey [:glandium] from comment #6)

Between a linux --disable-compile-environment and windows normal build, the same artifacts are identical (target.talos.tests.tar.gz, target.raptor.tests.tar.gz, target.jsreftest.tests.tar.gz, target.condprof.tests.tar.gz, target.awsy.tests.tar.gz), but there are more differences in the others:

• the mozinfo.json differences are more extensive. It feels like mozinfo.json could be an artifact of its own instead of being in (some) of the archives.
• target.reftests.tar.gz has differences in reftest.jsm because of a #ifdef MOZ_ENABLE_SKIA_PDF in the file (there's also a #ifdef XP_MACOSX, BTW, so that would be more differences yet on mac)
• target.mochitest.tests.tar.gz has more files on Linux.

All these three above are problems that will surface if moving to source checkouts instead.

Awesome work! Looks like toolchain tasks should generally work really well for these.

Whether we have source checkouts or not, we need a way to deal with platform specific differences, as well as any compiled binaries that need to be included.

Looking at https://firefox-source-docs.mozilla.org/build/buildsystem/mozinfo.html mozinfo looks like information that should come from the build anyway.

I'm definitely not familiar with any of the js, but I'm guessing the differences in reftest.jsm could be handled at runtime rather than build time.

Looking at mochitest in mozbuild files

• https://searchfox.org/mozilla-central/rev/dc4560dcaafd79375b9411fdbbaaebb0a59a93ac/layout/style/test/moz.build#139-146
• https://searchfox.org/mozilla-central/rev/dc4560dcaafd79375b9411fdbbaaebb0a59a93ac/toolkit/components/ctypes/tests/moz.build#29-32
  both of which look like things that should probably be packaged as part of the build.

(In reply to Mike Hommey [:glandium] from comment #6)

Between a linux --disable-compile-environment and windows normal build, the same artifacts are identical (target.talos.tests.tar.gz, target.raptor.tests.tar.gz, target.jsreftest.tests.tar.gz, target.condprof.tests.tar.gz, target.awsy.tests.tar.gz), but there are more differences in the others:

• the mozinfo.json differences are more extensive. It feels like mozinfo.json could be an artifact of its own instead of being in (some) of the archives.

We already upload mozinfo.json, and have for the past ~five years:

https://searchfox.org/mozilla-central/rev/dc4560dcaafd79375b9411fdbbaaebb0a59a93ac/toolkit/mozapps/installer/upload-files.mk#380

Guess we should file a bug to make tests pull in that artifact?

I'm going to work on this \o/

🎉🎉🎉🎉

:rstewart, will someone be continuing this work?

I am not aware of anyone who's planning on working on this in the near future.

The bug assignee didn't login in Bugzilla in the last 7 months.
:mhentges, could you have a look please?
For more information, please visit auto_nag documentation.