Closed
Bug 1380893
Opened 7 years ago
Closed 7 years ago
Stand up regular builds with sccache disabled
Categories
(Firefox Build System :: General, enhancement)
Firefox Build System
General
Tracking
(firefox57 wontfix, firefox61 fixed)
RESOLVED
FIXED
mozilla61
People
(Reporter: glandium, Assigned: gps)
References
Details
Attachments
(1 file, 1 obsolete file)
|
(deleted),
text/x-review-board-request
|
Details |
This is the opposite end of bug 1373460. Having spent some time looking at cache miss issues, rust build times and having seen first hand the large difference of build times between a build that has cache and one that doesn't, it now seems to me imperative that we set up builds that *never* use any form of caching, and track the build times for those, as they will more closely match the experience of clobber builds for local developers. That's something that's entirely lost in the noise of normal builds that, most of the time, get large cache hits (even when there are cache misses).
|
Assignee | |
Comment 1•7 years ago
|
||
How about standing up tasks that basically just do `mach build` and nothing else? No symbol dumping. No test archive generation. Maybe not even using mozharness. Just configure+make.
(Although, mozharness does have the perfherder integration today, so we may need mozharness in some limited capacity unless we want to move that code from desktop_build.py to the build system.)
|
Reporter | |
Comment 2•7 years ago
|
||
Fair point, the build times we measure are including those things, and they're just noise compared to how local builds perform.
|
|
Assignee | |
Updated•7 years ago
|
status-firefox57:
--- → wontfix
|
|
Assignee | |
Updated•7 years ago
|
Assignee: nobody → gps
Status: NEW → ASSIGNED
| Comment hidden (mozreview-request) |
|
||
Comment 4•7 years ago
|
||
| mozreview-review | ||
Comment on attachment 8944964 [details]
Bug 1380893 - Run build configuration that performs a clobber without caches;
https://reviewboard.mozilla.org/r/215104/#review220722
::: taskcluster/ci/build/macosx.yml:34
(Diff revision 1)
> - linux64-libdmg
> - linux64-llvm-dsymutil
> - linux64-rust-macos
> - linux64-sccache
>
> macosx64/opt:
macosx64-clobber/opt
Attachment #8944964 -
Flags: review+
| Comment hidden (mozreview-request) |
| Comment hidden (mozreview-request) |
|
|
Reporter | |
Comment 7•7 years ago
|
||
| mozreview-review | ||
Comment on attachment 8944964 [details]
Bug 1380893 - Run build configuration that performs a clobber without caches;
https://reviewboard.mozilla.org/r/215104/#review220770
::: build/mozconfig.common:25
(Diff revision 2)
> + # mozconfig.automation enables additional build steps. We don't care
> + # about those in a no caches build. So we tie them to the config. This
> + # isn't logically correct. But it works for our needs.
> -. "$topsrcdir/build/mozconfig.automation"
> + . "$topsrcdir/build/mozconfig.automation"
It seems to me skipping mozconfig.automation should be gated on something else. That seems like something desirable for many other jobs that currently include it. In fact, I'd go as far as saying the contents of that mozconfig should be set from taskcluster configs instead.
If you don't want to go that far, I'd just set the various environment for those jobs only, they will override the contents of mozconfig.automation.
::: build/mozconfig.common:29
(Diff revision 2)
> + # mozconfig.automation enables additional build steps. We don't care
> + # about those in a no caches build. So we tie them to the config. This
> + # isn't logically correct. But it works for our needs.
> -. "$topsrcdir/build/mozconfig.automation"
> + . "$topsrcdir/build/mozconfig.automation"
> -. "$topsrcdir/build/mozconfig.rust"
> -. "$topsrcdir/build/mozconfig.cache"
> + . "$topsrcdir/build/mozconfig.cache"
I'd rather see bug 1417421 fixed than hiding this before some environment variable.
::: taskcluster/taskgraph/transforms/job/common.py:20
(Diff revision 2)
> def docker_worker_add_workspace_cache(config, job, taskdesc, extra=None):
> """Add the workspace cache.
>
> ``extra`` is an optional kwarg passed in that supports extending the cache
> key name to avoid undesired conflicts with other caches."""
> + if job['worker'].get('disable-default-caches'):
should this instead be done in e.g. build_docker_worker_payload, avoiding any caches that would be set elsewhere in the future?
Attachment #8944964 -
Flags: review?(mh+mozilla)
|
||
Updated•7 years ago
|
Product: Core → Firefox Build System
|
|
Assignee | |
Updated•7 years ago
|
Attachment #8944964 -
Attachment is obsolete: true
| Comment hidden (mozreview-request) |
|
||
Comment 9•7 years ago
|
||
| mozreview-review | ||
Comment on attachment 8969727 [details]
Bug 1380893 - Add CI for plain builds;
https://reviewboard.mozilla.org/r/238532/#review246694
::: testing/mozharness/mozharness/mozilla/building/buildbase.py:1787
(Diff revision 1)
>
> + def ensure_upload_path(self):
> + env = self.query_mach_build_env()
>
> + # Some Taskcluster workers don't like it if an artifacts directory
> + # is defined by no artifacts are uploaded. Guard against this by always
by no -> but no
Attachment #8969727 -
Flags: review+
|
|
||
Updated•7 years ago
|
Attachment #8969727 -
Flags: review?(core-build-config-reviews)
| Comment hidden (mozreview-request) |
|
|
Assignee | |
Updated•7 years ago
|
Attachment #8969727 -
Flags: review?(core-build-config-reviews)
|
||
Comment 11•7 years ago
|
||
Pushed by gszorc@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3accdb3ed507
Add CI for plain builds; r=mshal
|
||
Comment 12•7 years ago
|
||
| bugherder | ||
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
status-firefox61:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla61
You need to log in
before you can comment on or make changes to this bug.
Description
•