Closed Bug 1259789 Opened 9 years ago Closed 7 years ago

build-backend time has regressed

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: chmanchester, Unassigned)

References

Details

Back in https://bugzilla.mozilla.org/show_bug.cgi?id=1203266#c4 I'm claiming 190ms is ~3% of build-backend time on my laptop, which would make the overall time ~6 seconds. This is consistent with my recollection, because I was using this number as a benchmark to make sure I wasn't regressing it. Today, build-backend on my laptop takes over 8 seconds. For slower platforms I would expect this to be quite noticeable. A variety of things could have done this in the last 5-6 months. I'll investigate as soon as I find an opportunity.
hg bisect says: The first bad revision is: changeset: 288747:bd4a4166af52 user: Gregory Szorc <gps@mozilla.com> date: Mon Mar 14 21:06:57 2016 -0700 summary: Bug 1253436 - Write out a machine readable file with binaries metadata; r=glandium But the baseline before that is still 7s, so there may be an earlier regression in there.
Blocks: 1253436
FWIW, I just found that dumping all-tests.json takes a third of the overall time spent running mach build-backend on my machine.
Not surprising. The file has ballooned over the course of a year. It's now 50+ MB. Using a faster JSON serializer like ujson should make things better. But I'd prefer we reduce the size of the file and/or break it up somehow.
There are some highly redundant aspects to the format we're using all-tests.json, we could probably make it a lot smaller. This also accounts for a large part of the remaining overhead for running tests (it's now more expensive than installing tests). But there's a specific regression here.
From bisecting this again this morning, bug 1277424 adds about another 3s to build-backend on my laptop, I think due to the huge number of tests and support files added in that commit.
No longer blocks: 1253436
Depends on: 1253436, webgl-104-ci
Product: Core → Firefox Build System
Various improvements were implemented in dependent bugs, I'm not planing to pursue anything further here.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.