hg outgoing can take a long time when re-building taskgraph in |mach try|
Categories
(Firefox Build System :: General, enhancement, P3)
Tracking
(Not tracked)
People
(Reporter: bugzilla, Unassigned)
Details
(Keywords: in-triage)
Maybe this is related to Mercurial 5.0.0, which I was upgraded to when I ran mach bootstrap
earlier today?
I gave up after waiting 15 minutes:
$ ./mach try chooser
Task configuration changed, generating target task set
hg
is consuming a bunch of CPU. Its command line:
c:\Users\foo\mozilla-build\python\Scripts\hg.exe outgoing -r . --quiet --template "{file_adds % '\n{file}'}{file_mods % '\n{file}'}" default
Reporter | ||
Updated•6 years ago
|
Reporter | ||
Updated•6 years ago
|
Comment 1•6 years ago
|
||
Have you confirmed that the upgrade to 5.0 breaks it? If not, can you try to rollback the version of mercurial to 4.9? I just manually updated to 5.0 and am hitting all sorts of problems with the firefoxtree
and push-to-try
extensions. Either the upgrade to 5.0 was an accident or maybe just premature.
Connor, do you know if we meant to be updating people to 5.0 already? Do our extensions (push-to-try
, firefoxtree
, etc) all need to be Python 3 compatible before we can update people?
Reporter | ||
Comment 2•6 years ago
|
||
Hmm, maybe it's not related to 5.0 after all... I downgraded to 4.9.1 and am still experiencing this problem.
Comment 3•6 years ago
|
||
On Windows, bootstrap just pip install's mercurial, without a specific version being specified, so whatever is on pypi at the time of bootstrap is installed. On other platforms, packages are relied upon, so it depends when the respective packages are updated for new versions of mercurial. In any case, bootstrap doesn't limit what version is going to be used (and apart from the pip case, can't really)
Reporter | ||
Comment 4•6 years ago
|
||
Could this have something to do with bug 1543872? Edit: Hmmm, backing that out didn't appear to do anything...
Reporter | ||
Comment 5•6 years ago
|
||
Okay, I figured out what my particular problem was, though I have no idea why I didn't start seeing this until now:
The default path for my working directory pointed to an old revision of mozilla-aurora
, so the hg outgoing
command was taking its sweet time generating a diff between my current working copy based off of central, and that default repo.
Updating my .hg/hgrc
such that the default correctly points to central fixed my problem. What that doesn't explain is why this only started now.
Furthermore, this implies that mach try
isn't going to work very well when anybody's working copy is set to a head that is based off of a different upstream from the default.
Comment 6•6 years ago
|
||
Ok, looks like my mercurial 5.0 problems are related to evolve
and not firefoxtree
/push-to-try
. Though I'm surprised they pushed 5.0 to pypi already, it looks like it hasn't yet been officially released. Maybe our bootstrap script should use "^4.9" as a version specifier so we don't automatically pick up major upgrades until we explicitly tell it to.
Back to the issue at hand..
The get_outgoing_files
function is called by the taskgraph module (when re-generating tasks), and not directly by mach try
. This distinction doesn't really matter to the end user, but useful for context. I'm not really sure what we can do about this tbh, and fwiw I've previously pushed from very old heads without issue (though this was Linux).
Updated•6 years ago
|
Comment 7•6 years ago
|
||
The call to hg outgoing
originates here:
https://searchfox.org/mozilla-central/source/taskcluster/taskgraph/util/perfile.py#58
Updated•6 years ago
|
Reporter | ||
Comment 8•6 years ago
|
||
I am by no means an expert at hg revsets, so YMMV, but assuming that firefoxtree
is installed, this command will dump the repo that the current changes are built atop:
hg log --pager never -r 'p1(first(ancestors(.) and not public())) or .' --template "{get(namespaces, 'fxtrees')}\n"
Maybe the output from that could be used for the upstream
parameter instead of default
?
Comment 9•6 years ago
|
||
Good idea. We can't rely on firefoxtree
being installed, but we could continue to use default
as a fallback in that case. This is now an issue in mozversioncontrol (which doesn't have its own component).
Updated•6 years ago
|
Updated•2 years ago
|
Description
•