Closed
Bug 987267
Opened 11 years ago
Closed 10 years ago
use gzip compression for hg bundles
Categories
(Release Engineering :: General, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: taras.mozilla, Assigned: rail)
References
Details
Attachments
(1 file, 1 obsolete file)
(deleted),
patch
|
catlee
:
review+
rail
:
checked-in+
|
Details | Diff | Splinter Review |
hg operations are gonna take as long or longer than builds once we implement caching on windows. According to glandium's stats, hg unbundling only happens on ~3% of the builds, but it does lead to some surprisingly-slow build times.
Simply zipping up the whole repo ends in a 3x speed up, switching to gzip bundles is a 1.4x speedup. Lets swap to gzip bundles asap(or just hg clone --uncompressed on windows which would be fastest)
++ mkdir mozilla-central
++ cd mozilla-central
++ hg init .
++ hg unbundle ../mozilla-central.hg
adding changesets
adding manifests
adding file changes
added 174831 changesets with 983111 changes to 153611 files
(run 'hg update' to get a working copy)
real 31m59.826s
user 0m0.015s
sys 0m0.015s
++ cd mozilla-central
++ hg init .
++ hg unbundle ../mozilla-central.gz.bundle
adding changesets
adding manifests
adding file changes
added 174831 changesets with 983111 changes to 153611 files
(run 'hg update' to get a working copy)
real 22m59.116s
user 0m0.015s
sys 0m0.030s
++ rm -fR mozilla-central
++ unzip -q mozilla-central.zip
real 11m19.762s
user 0m0.015s
sys 0m0.000s
1.1G mozilla-central.zip
914M mozilla-central.7z
790M mozilla-central.gz.bundle
631M mozilla-central.hg
Assignee | ||
Comment 1•11 years ago
|
||
Switching to gzip is easy. We'll need to regenerate the bundles (see https://wiki.mozilla.org/ReleaseEngineering/Merge_Duty/Steps#Create_new_hg_bundles)
zipping .hg would require changes in hg-tool.py
Attachment #8395825 -
Flags: review?(catlee)
Assignee | ||
Comment 2•11 years ago
|
||
err, wrong position
Attachment #8395825 -
Attachment is obsolete: true
Attachment #8395825 -
Flags: review?(catlee)
Attachment #8395828 -
Flags: review?(catlee)
Updated•11 years ago
|
Attachment #8395828 -
Flags: review?(catlee) → review+
Assignee | ||
Comment 3•11 years ago
|
||
Comment on attachment 8395828 [details] [diff] [review]
gzip.bundle.diff
https://hg.mozilla.org/build/tools/rev/44faaa1a3937
Attachment #8395828 -
Flags: checked-in+
Assignee | ||
Comment 4•11 years ago
|
||
Regenerated the bundles.
Reporter | ||
Comment 5•11 years ago
|
||
(In reply to Rail Aliiev [:rail] from comment #4)
> Regenerated the bundles.
<3
Any reason to keep this open? We can pursue other options in another bug another time.
Updated•10 years ago
|
Assignee: nobody → rail
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Summary: hg bundles suck → use gzip compression for hg bundles
Comment 6•10 years ago
|
||
23 minutes to do an unbundle from a gzipped bundle and 11:20 to unzip?! My computer can do a gzip unbundle in 8 minutes. And 4 of those should go away with bundle2 (I'd like to add cache data into bundles so clients don't need to regenerate it).
Updated•7 years ago
|
Component: General Automation → General
You need to log in
before you can comment on or make changes to this bug.
Description
•