We have a number of miscellaneous Python scripts in the tree that perform custom build actions. e.g. purge manifests, nsinstall, xpccheck.py. Many of these are in /build and /config. I would like to unite all of the scripts under a common directory and provide a common mechanism for invoking them from make files. Benefits: * We encourage build actions to be written as importable Python modules, not as scripts. This fosters better/easier testing and reusability. * If we put all scripts in a package on sys.path, this should eliminate some of the copies in /js/src. * Easier to employ pymake magic to universally inline scripts (as opposed to running a new process/interpreter). * Help cut down on pythonpath.py abuse. Patch forthcoming.

I created python/mozbuild/mozbuild/action to hold scripts/modules that can be used as build actions. I moved the purge manifests and xpcshell manifest auditing scripts there. I created a "py_action" callable in config.mk that simply invokes said scripts from the well-defined location. This seems to "just work." I stopped short of employing pymake % magic. Seeking sign-off from multiple build peers because this idea may come across as somewhat radical.

I was playing around with pymake % magic locally and doing it right will be a little bit of effort. Currently, the sys.path of pymake doesn't contain the virtualenv. However, you can define $(PYCOMMANDPATH) to be a directory to append to sys.path. We currently use $(topsrcdir)/config in config.mk. Unfortunately, it looks like pymake internally treats this variable as a string, so assigning multiple word-delimited paths to this variable does not work. Furthermore, since $(topsrcdir) resolves differently in /js/src, we won't be able to reference /python/mozbuild easily (or is there a backdoor somewhere - I can't recall). I think what I'm trying to say is that I think we need to properly hook pymake up the virtualenv and that this is a prerequisite to native/% execution of build actions from pymake.

I agree with this. The Python native command stuff was implemented before we had a virtualenv, so it didn't really take that into account. PYCOMMANDPATH works with either space or os.sep-separated paths: http://mxr.mozilla.org/mozilla-central/source/build/pymake/tests/native-pycommandpath-sep.mk http://mxr.mozilla.org/mozilla-central/source/build/pymake/tests/native-pycommandpath.mk ...but ideally it'd just be using the virtualenv. That might be a PITA, I don't know.

Comment on attachment 772781 [details] [diff] [review] Establish unified directory for Python build actions (splinter annoyingly hides changes with no diff)

Comment on attachment 772781 [details] [diff] [review] Establish unified directory for Python build actions Review of attachment 772781 [details] [diff] [review]: ----------------------------------------------------------------- Seems fine. I could bikeshed about the naming of the directory, but I don't care that much.

comm-central bustage fix: https://hg.mozilla.org/comm-central/rev/c5397c5f0faf