Yeah, I know all about the multiple duplicate bugs to automate it, but they're stalled and People Are Dying. I'd dismiss slice:~/mc/mozilla philor$ time hg push -f -v try pushing to ssh://hg.mozilla.org/try/ running ssh hg.mozilla.org "hg -R try/ serve --stdio" searching for changes 1 changesets found remote: adding changesets remote: adding manifests remote: adding file changes remote: added 1 changesets with 4 changes to 4 files (+1 heads) remote: Looks like you used try syntax, going ahead with the push. remote: If you don't get what you expected, check http://people.mozilla.org/~lsblakk/trychooser/ for help with building your trychooser request. remote: Thanks for helping save resources, you're the best! remote: Trying to insert into pushlog. remote: Please do not interrupt... remote: Inserted into the pushlog db successfully. real 66m50.535s user 0m1.723s sys 0m0.463s as just my crappy connection, but I heard about someone else's 40 minute push the other day, and 5, 10, or 15 minute pushes (mine prior to that 66 minute one was 7 minutes) are constantly being complained-about. Please do whatever stats gathering that was that you used to do when we used to trim it regularly, schedule a downtime, and just throw it away and start over with a nice new mozilla-central.

(philor: Unstated, but I assume you are only seeing this slow performance on try repo. If you are seeing this with other repos, we have a wider hg.m.o problem to address.) Instead of deleting the try repo, lets do what worked earlier this year: schedule downtime to: * rename try -> try.broken * create new try repo from m-c ...and then I can pull stats data from try.broken when I get back from vacation.

Do we need to close anything other than Try?

Yes, I'm only seeing it and only hearing about it for Try. And I can't imagine why it would require closing anything but Try, but I didn't even know we had done a reset earlier this year - what did we do then?

according to our notes, we don't have to close anything other than try.

If the issue is due to high number of heads, I wonder if it would be possible to make a script that once a week hg strips all heads older than a certain amount of time (like a month). Going through heads would be easy, filtering them by date would be easy, but I wonder if there's a way to get the first changeset that inited a specified branch to strip it. Alternatively I think just closing heads may work similarly. This would allow to cleanup without the need to close the tree, while maintaining perf constant.

(In reply to Marco Bonardo [:mak] from comment #5) > If the issue is due to high number of heads, I wonder if it would be > possible to make a script that... Bug 554656 / bug 633161 are looking into this, but have been stalled for a while: (In reply to Phil Ringnalda (:philor) from comment #0) > Yeah, I know all about the multiple duplicate bugs to automate it, but > they're stalled and People Are Dying.

Taking advantage of the tree closure due to bug 697374 this morning, Amy moved the old try repo aside (to address joduinn's comment #1), and then re-cloned the try repo from m-c. We're currently investigating how this is going to affect using local mirrors with our build systems (bug 665021), but try *should* be usable again once the trees reopen.

(In reply to John O'Duinn [:joduinn] from comment #1) > (philor: Unstated, but I assume you are only seeing this slow performance on > try repo. If you are seeing this with other repos, we have a wider hg.m.o > problem to address.) > > > Instead of deleting the try repo, lets do what worked earlier this year: > > schedule downtime to: > * rename try -> try.broken > * create new try repo from m-c I've moved the existing repo to /repo/hg/mozilla/try-bug696682 and run /repo/hg/scripts/reset_try.sh. Bug 697457 has been opened to fix the interaction between the (not yet in use) mirrors.

Aside from the deletion of the bad repo when joduinn is done with it, is there any work left here?

(In reply to John Ford [:jhford] from comment #9) > Aside from the deletion of the bad repo when joduinn is done with it, is > there any work left here? Devs are reporting that new commits aren't appearing in hgweb, although I can confirm that the commits *are* making it into the try repo. I've pinged IT about tracking down what needs to be flushed in hgweb.

Changing to blocker due to comment 10 + inability to use TBPL with try: https://tbpl.mozilla.org/?tree=Try

(In reply to Chris Cooper [:coop] from comment #10) > Devs are reporting that new commits aren't appearing in hgweb In particular, http://hg.mozilla.org/try/pushloghtml is basically blank. (compare to http://hg.mozilla.org/mozilla-central/pushloghtml )

OK, since try is still closed, I've asked IT to try re-cloning Try from m-c, making sure that the pushlog and all the hooks are in place from the get-go. Apologies to devs that have pushed-to-try in the interim, you'll need to push again once we reopen the tree.

we ended up cloning again due to a busted pushlog. If you have pushed to try today before now, you might need to push again due to the repository being removed. I just tested pushing to try results in an entry in /try/pushlog and that I was emailed on the start of the build jobs. That email comes from buildbot, so I take that as a sign that things are working.

(In reply to John Ford [:jhford] from comment #14) > we ended up cloning again due to a busted pushlog. If you have pushed to > try today before now, you might need to push again due to the repository > being removed. > > I just tested pushing to try results in an entry in /try/pushlog and that I > was emailed on the start of the build jobs. That email comes from buildbot, > so I take that as a sign that things are working. Yes, second re-clone did the trick.