Closed
Bug 921070
Opened 11 years ago
Closed 11 years ago
Likely race condition building WebIDL with MOZ_PSEUDO_DERECURSE
Categories
(Firefox Build System :: General, defect)
Firefox Build System
General
Tracking
(Not tracked)
RESOLVED
FIXED
mozilla27
People
(Reporter: gps, Assigned: gps)
References
Details
Attachments
(1 file, 1 obsolete file)
(deleted),
patch
|
glandium
:
review+
|
Details | Diff | Splinter Review |
I've encountered the following error during export with MOZ_PSEUDO_RECURSE a few times now. I suspect we have a race condition building WebIDL, possibly due to the dom/bindings/Makefile being invoked multiple times during traversal.
I've seen this over a few days, so it's not the unified compiling patch that busted it.
0:42.96 Generating global WebIDL files
0:46.31 GeneratedAtomList.h hasn't changed - not touching it
0:46.31 Generating PrototypeList.h
0:46.32 RegisterBindings.h hasn't changed - not touching it
0:46.32 RegisterBindings.cpp hasn't changed - not touching it
0:46.32 UnionTypes.h hasn't changed - not touching it
0:46.32 Generating UnionTypes.cpp
0:46.32 UnionConversions.h hasn't changed - not touching it
0:46.32 Generating WebIDL bindings
0:46.78 Traceback (most recent call last):
0:46.78 File "/Users/gps/src/firefox/config/pythonpath.py", line 56, in <module>
0:46.78 main(sys.argv[1:])
0:46.78 File "/Users/gps/src/firefox/config/pythonpath.py", line 48, in main
0:46.78 execfile(script, frozenglobals)
0:46.78 File "/Users/gps/src/firefox/dom/bindings/BindingGen.py", line 98, in <module>
0:46.78 main()
0:46.78 File "/Users/gps/src/firefox/dom/bindings/BindingGen.py", line 58, in main
0:46.78 config = cPickle.load(f)
0:46.78 EOFError
0:46.85 make[6]: *** [.BindingGen] Error 1
0:46.85 make[5]: *** [webidl] Error 2
0:46.85 make[5]: *** Waiting for unfinished jobs....
0:47.36 GeneratedAtomList.h hasn't changed - not touching it
0:47.36 PrototypeList.h hasn't changed - not touching it
0:47.36 RegisterBindings.h hasn't changed - not touching it
0:47.36 RegisterBindings.cpp hasn't changed - not touching it
0:47.36 UnionTypes.h hasn't changed - not touching it
0:47.36 UnionTypes.cpp hasn't changed - not touching it
0:47.36 UnionConversions.h hasn't changed - not touching it
0:47.36 Generating WebIDL bindings
0:55.72 make[4]: *** [config/makefiles/precompile/export] Error 2
0:55.72 make[3]: *** [export] Error 2
0:55.72 make[2]: *** [default] Error 2
0:55.72 make[1]: *** [realbuild] Error 2
0:55.73 make: *** [build] Error 2
0:55.74 362 compiler warnings present.
Comment 1•11 years ago
|
||
I've seen this too and assumed it was a problem with the unified build patches, since a clean tree built (once!) with no problems.
Comment 2•11 years ago
|
||
Note that dom/moz.build has:
97 # bindings/test is here, because it needs to build after bindings/, and
98 # we build subdirectories before ourselves.
as in there is a definite ordering dependency there, because otherwise the make in test/ will try to make in the parent and try to rebuild the pickle which is in the process of being written still, and then you race on writes to it and things are bad. Are we not enforcing this ordering dependency with MOZ_PSEUDO_RECURSE? Can we remove the ordering dependency somehow?
Assignee | ||
Comment 3•11 years ago
|
||
Yeah, pretty sure the export target is getting invoked twice during traversal.
From config/makefiles/precompile:
-------------
export:: ipdl webidl xpidl
webidl:
$(call SUBMAKE,webidl,$(DEPTH)/dom/bindings)
-------------
From dom/bindings:
---------
EXPORTS_GENERATED_FILES := $(exported_binding_headers) $(exported_generated_events_headers)
EXPORTS_GENERATED_DEST := $(DIST)/include/$(binding_include_path)
EXPORTS_GENERATED_TARGET := export
INSTALL_TARGETS += EXPORTS_GENERATED
webidl:: $(generated_header_files)
---------
With MOZ_PSEUDO_DERECURSE, we'll attempt to generate a bunch of header files in parallel, which likely leads to this race condition.
Now that we've inverted tiers, the precompile make file could likely die.
Assignee | ||
Updated•11 years ago
|
Summary: Likely race condition building WebIDL with MOZ_PSEUDO_RECURSE → Likely race condition building WebIDL with MOZ_PSEUDO_DERECURSE
Assignee | ||
Comment 4•11 years ago
|
||
Let's just slowly kill the precompile make file. That make file was made
less useful by tier inversion and even less useful by
MOZ_PSEUDO_DERECURSE.
This will likely slow down non-MOZ_PSEUDO_DERECURSE builds a bit. But
since we're about to make MOZ_PSEUDO_DERECURSE=export-only the default
and I assume we'll soon fix the pymake assertion, meh.
Attachment #810682 -
Flags: review?(mh+mozilla)
Assignee | ||
Updated•11 years ago
|
Assignee: nobody → gps
Assignee | ||
Comment 5•11 years ago
|
||
Go big or go home. I completely killed the precompile tier. I believe I
nuked all references. Local builds with and without MOZ_PSEUDO_DERECURSE
seem to work. non-MOZ_PSEUDO_DERECURSE are a little slower now that
{xpidl, ipdl, webidl} are no longer built side-by-side. But that will be
fixed soon enough.
https://tbpl.mozilla.org/?tree=Try&rev=4ed85b591abe
Attachment #810694 -
Flags: review?(mh+mozilla)
Assignee | ||
Updated•11 years ago
|
Attachment #810682 -
Attachment is obsolete: true
Attachment #810682 -
Flags: review?(mh+mozilla)
Assignee | ||
Comment 6•11 years ago
|
||
Same patch, different base revision:
https://tbpl.mozilla.org/?tree=Try&rev=9806ef92b10a
Status: NEW → ASSIGNED
Updated•11 years ago
|
Attachment #810694 -
Flags: review?(mh+mozilla) → review+
Assignee | ||
Comment 7•11 years ago
|
||
Comment 8•11 years ago
|
||
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla27
Updated•7 years ago
|
Product: Core → Firefox Build System
You need to log in
before you can comment on or make changes to this bug.
Description
•