The try repo is too slow to clone from and to fetch the pushlog from. Our solution up to this point is to reset the repository periodically, but this is buying us less and less time. I think it's worth investigating exactly why hg / pushlog is slow here, and seeing if there's anything we can do to speed it up.

I've always assumed hg just gets slower as the number of heads increases. Maybe there's an O(N^2) algorithm on the number of heads in there somewhere. The /pushlog ATOM feed shouldn't hit the hg repo directly, so it shouldn't be affected if that's the case, but /pushloghtml does hit the repo to look up commit messages etc.

Isn't this just a dupe of bug 554656? At least that has a bunch of discussion.

(In reply to comment #1) > I've always assumed hg just gets slower as the number of heads increases. Maybe > there's an O(N^2) algorithm on the number of heads in there somewhere. The > /pushlog ATOM feed shouldn't hit the hg repo directly, so it shouldn't be > affected if that's the case, but /pushloghtml does hit the repo to look up > commit messages etc. TBPL hits /pushloghtml, so that explains why it's slow. (In reply to comment #2) > Isn't this just a dupe of bug 554656? At least that has a bunch of discussion. If automatically pruning fixes the slowness, great. If we can fix it another way, that would be fine as well.

Collapsing even further, for the long term issue. Way too many bugs on this...

I'm unduping this because this bug is about fixing the underlying issue.

It seems unlikely that the problem is related to the pushlog db. Hitting the url for an individual revision is pretty sluggish, which indicates the hg repo itself is slow. e.g. time wget -O/dev/null http://hg.mozilla.org/try/rev/7c68d19133b4 --2010-06-04 17:08:09-- http://hg.mozilla.org/try/rev/7c68d19133b4 Resolving hg.mozilla.org... 10.2.74.67 Connecting to hg.mozilla.org|10.2.74.67|:80... connected. HTTP request sent, awaiting response... 200 Script output follows Length: unspecified [text/html] Saving to: `/dev/null' [ <=> ] 13,497 18.8K/s in 0.7s 2010-06-04 17:08:27 (18.8 KB/s) - `/dev/null' saved [13497] wget -O/dev/null http://hg.mozilla.org/try/rev/7c68d19133b4 0.00s user 0.00s system 0% cpu 18.493 total

On a full clone of try... hg 1.5: time hg heads > /dev/null hg heads > /dev/null 0.26s user 0.03s system 93% cpu 0.305 total hg 1.2: time hg heads > /dev/null hg heads > /dev/null 11.20s user 0.75s system 99% cpu 12.068 total

Sounds like updating hg (bug 551015) would make this better then. We still need to sort out why it breaks our hgwebdir setup though...

This will largely be addressed by bug 554656 and bug 589885 so this bug has no actual work to be done for it.