(not currently active) Ted Mielczarek Reporter
	Description • 14 years ago

Since bug 581938 merged the pymake native-commands branch to mozilla-central, we can now start hacking our build system to invoke Python as native pymake commands instead of spawning interpreter processes. Pymake now keeps a process pool around, and native commands simply execute Python in one of the processes instead of spawning a new process, so there should be way less overhead.

Pymake native commands are pretty simple. You add a % character in front of the command, and the rest of the line is parsed as <module name> <method> <args>, so something like:
%foo.bar doSomething 1 2 3

Would try to import the Python module foo.bar, then invoke the method doSomething from it with one argument, [1,2,3].

The module search path can be set by setting PYCOMMANDPATH as a makefile variable. Multiple paths can be added separated by spaces or the native pathsep character.

Anything we add should obviously be ifdef .PYMAKE so that gmake builds continue to work.

There are some examples of native commands in pymake's builtins:
http://hg.mozilla.org/users/bsmedberg_mozilla.com/pymake/file/e50be3234e39/pymake/builtins.py

And there are a few examples of usage in the test suite:
http://hg.mozilla.org/users/bsmedberg_mozilla.com/pymake/file/e50be3234e39/tests/native-simple.mk
http://hg.mozilla.org/users/bsmedberg_mozilla.com/pymake/file/e50be3234e39/tests/native-pycommandpath.mk
http://hg.mozilla.org/users/bsmedberg_mozilla.com/pymake/file/e50be3234e39/tests/native-pycommandpath-sep.mk

	(not currently active) Ted Mielczarek Reporter
	Comment 1 • 14 years ago

Ok, that's all the obvious deps I can think of. Ideal candidates are things that we're running one or more times per Makefile during the normal build phase. Stuff like packaging etc, while nice, is less of a win because it doesn't get run as much.

Other things to consider:
1) Hunt down any remaining Perl scripts that get run during the build, rewrite in Python, run as native commands (FTW)
2) Remove complicated shell control structures from make commands, rewrite as small Python scripts, run as native commands

These should both add up to wins in terms of maintenance and build speed.