Closed
Bug 748940
Opened 13 years ago
Closed 13 years ago
HG outage Apr 25, 2012
Categories
(Developer Services :: General, task)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: rail, Assigned: bkero)
References
Details
(Whiteboard: [buildduty][outage])
We're seeing some 500 errors trying to clone from hg.m.o:
======== Started clone build tools failed (results: 5, elapsed: 0 secs) (at 2012-04-25 13:10:09.913842) =========
hg clone http://hg.mozilla.org/build/tools tools
.....
abort: HTTP Error 500: Internal Server Error
program finished with exit code 255
it was very slow ~1h ago.
Also there was an email from
According to monitor 'hg.mozilla.org', the http service on 'hg.mozilla.org' has
not been working as specified since 2012-04-25 13:12:13.
Message: Timeout while connecting
.....
Regards,
- the WatchMouse Team
Updated•13 years ago
|
Whiteboard: [buildduty][outage]
Assignee | ||
Comment 1•13 years ago
|
||
This happens from time to time, and the short term solution is to restart the http daemon on the web heads.
Service was restored for this within 5 minutes of the report reaching me on IRC.
I'm uncertain of the underlying cause of this and will need to dig deeper, perhaps into the hgweb code itself in order to determine a root cause.
Since service is restored, I'm going to bring this down to normal importance and use it as a bug to track determining the root cause of failure.
Assignee: server-ops → bkero
Severity: critical → normal
Comment 2•13 years ago
|
||
bkero: trees remain closed because of hg lossage. Details in bug#748939. Not sure if these are same hg issues, so not sure if I should close as DUP or raise to blocker.
Assignee | ||
Comment 3•13 years ago
|
||
joduinn: this seems to be related to a missing file related to http://build.mozilla.org, the hg 404 was a red herring. More details as they're revealed in #build.
Comment 5•13 years ago
|
||
(In reply to Ben Kero [:bkero] from comment #3)
> joduinn: this seems to be related to a missing file related to
> http://build.mozilla.org, the hg 404 was a red herring. More details as
> they're revealed in #build.
bkero: ah, good to know. Can you (or whomever knows what happened) update this bug when problem understood.
Meanwhile, I'm happy to see trees open again, so this can stay normal, and its fine to DUP/CLOSE the other blocker bug#748939.
Updated•13 years ago
|
Component: Server Operations → Server Operations: Developer Services
QA Contact: phong → shyam
Reporter | ||
Comment 6•13 years ago
|
||
404 errors were not related to HG, see bug 748996 for the details.
Assignee | ||
Updated•13 years ago
|
Assignee: bkero → server-ops
Status: NEW → RESOLVED
Closed: 13 years ago
Component: Server Operations: Developer Services → Server Operations
QA Contact: shyam → phong
Resolution: --- → FIXED
Updated•13 years ago
|
Assignee: server-ops → bkero
Component: Server Operations → Server Operations: Developer Services
QA Contact: phong → shyam
Updated•10 years ago
|
Component: Server Operations: Developer Services → General
Product: mozilla.org → Developer Services
You need to log in
before you can comment on or make changes to this bug.
Description
•