Closed
Bug 585010
Opened 14 years ago
Closed 8 years ago
[meta] start using pymake native commands in the build system
Categories
(Firefox Build System :: General, defect)
Firefox Build System
General
Tracking
(Not tracked)
RESOLVED
WONTFIX
People
(Reporter: ted, Unassigned)
References
(Blocks 1 open bug)
Details
(Whiteboard: [pymake])
Since bug 581938 merged the pymake native-commands branch to mozilla-central, we can now start hacking our build system to invoke Python as native pymake commands instead of spawning interpreter processes. Pymake now keeps a process pool around, and native commands simply execute Python in one of the processes instead of spawning a new process, so there should be way less overhead. Pymake native commands are pretty simple. You add a % character in front of the command, and the rest of the line is parsed as <module name> <method> <args>, so something like: %foo.bar doSomething 1 2 3 Would try to import the Python module foo.bar, then invoke the method doSomething from it with one argument, [1,2,3]. The module search path can be set by setting PYCOMMANDPATH as a makefile variable. Multiple paths can be added separated by spaces or the native pathsep character. Anything we add should obviously be ifdef .PYMAKE so that gmake builds continue to work. There are some examples of native commands in pymake's builtins: http://hg.mozilla.org/users/bsmedberg_mozilla.com/pymake/file/e50be3234e39/pymake/builtins.py And there are a few examples of usage in the test suite: http://hg.mozilla.org/users/bsmedberg_mozilla.com/pymake/file/e50be3234e39/tests/native-simple.mk http://hg.mozilla.org/users/bsmedberg_mozilla.com/pymake/file/e50be3234e39/tests/native-pycommandpath.mk http://hg.mozilla.org/users/bsmedberg_mozilla.com/pymake/file/e50be3234e39/tests/native-pycommandpath-sep.mk
Reporter | ||
Comment 1•14 years ago
|
||
Ok, that's all the obvious deps I can think of. Ideal candidates are things that we're running one or more times per Makefile during the normal build phase. Stuff like packaging etc, while nice, is less of a win because it doesn't get run as much. Other things to consider: 1) Hunt down any remaining Perl scripts that get run during the build, rewrite in Python, run as native commands (FTW) 2) Remove complicated shell control structures from make commands, rewrite as small Python scripts, run as native commands These should both add up to wins in terms of maintenance and build speed.
Whiteboard: [pymake]
Reporter | ||
Updated•13 years ago
|
Blocks: build-perf
Reporter | ||
Updated•8 years ago
|
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WONTFIX
Updated•7 years ago
|
Product: Core → Firefox Build System
You need to log in
before you can comment on or make changes to this bug.
Description
•