Closed Bug 852249 Opened 12 years ago Closed 12 years ago

Make pymake's includedepfiles use a dumber parser

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: ted, Assigned: ted)

References

Details

Attachments

(1 file)

If we drop mddepend.pl, that means we force make (or pymake) to parse a possibly hundreds of thousands of lines of .pp files per directory instead of roughly one. Pymake is really slow at this. Pymake has an "includedeps" feature that's indended to be used for this purpose, but it goes through the normal makefile parser so it's still slow. This patch makes includedeps use a really stupid parser that's similar to what mddepend.pl does. This is still not quite fast enough to replace mddepend.pl, possibly because pymake is bad at stat'ing files or something. We should figure that out too.
Blocks: 462463
How much slower is "really slow"? Seems like the build system is pretty broken right now, with pretty painful penalties for landing, bustage, clobber, waiting, etc. (As well as everyone's local builds affected from the CLOBBER ripples.) "Slow but working" might be a net-win.
(In reply to Ted Mielczarek from comment #0) > This is still not quite fast enough to replace mddepend.pl, possibly because > pymake is bad at stat'ing files or something. We should figure that out too. Well, one possibility is that mddepend.pl is optimised to stat as few files as possible - for each depending file, it checks all the dependent files that it's already cached first, so for instance once it stats nscore.h then it quickly realises that everything needs to be rebuilt.
(In reply to Justin Dolske [:Dolske] from comment #3) > How much slower is "really slow"? Seems like the build system is pretty > broken right now, with pretty painful penalties for landing, bustage, > clobber, waiting, etc. (As well as everyone's local builds affected from the > CLOBBER ripples.) "Slow but working" might be a net-win. I saw a slow down of 4 minutes in a single directory.
Comment on attachment 726283 [details] [diff] [review] Make pymake's includedepfiles use a dumber parser I made a couple changes to make it work on Windows, but other than that it looks good. r=me
Attachment #726283 - Flags: review+
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: