Closed Bug 599302 Opened 14 years ago Closed 14 years ago

cb-sea-miniosx03, cb-sea-miniosx04, cb-seamonkey-win32-01 are AWOL

Categories

(mozilla.org Graveyard :: Server Operations, task)

task
Not set
major

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: kairo, Assigned: phong)

References

Details

Callek told me there have been some hiccups with SeaMonkey build machines yesterday, which even led to rebooting the buildmaster, and now cb-sea-miniosx03, cb-sea-miniosx04, and cb-seamonkey-win32-01 are AWOL. All of them don't respond to trying to connect. I hope Callek didn't halt the minis when he wanted to reboot them, but they should just come back after a manual reboot. Not sure how easily IT can do that, though, might need a colo trip. :( As for cb-seamonkey-win32-01, this is on the Parallels host, and phong bumped RAM for 4 VMs on that yesterday, maybe this, erm, interesting virtualization technology now shuts down or freezes VMs when RAM gets tight somewhere. We might have had the same problem with cb-seamonkey-linuxmaster-01 at night when Callek looked into this and got phong to reboot that one.
I don't think the host can handle giving these VMs 3 GB of ram.
(In reply to comment #1) > I don't think the host can handle giving these VMs 3 GB of ram. Hmm, then it sounds like we have a problem we need to take care of. And that makes me cry again for those build machines I requested a year ago (not your fault we don't have them yet, but still pulling my nerves). ;-) I guess it might make sense to get that Windows one up again and shut down one of the Linux ones (cb-seamonkey-linux-03?) for the moment, but I need to run, so can't assist with that, perhaps Callek can, else let's do that next week. The Minis should be unaffected by all that, I hope we can get them back independently.
Assignee: server-ops → phong
Flags: colo-trip+
I brought the Windows VM only and it blew everything up. I had to restart everything and I left linux-03 off for now.
(In reply to comment #3) > I brought the Windows VM only and it blew everything up. I had to restart > everything and I left linux-03 off for now. That had the effect of all but one slaves being so confused that they did think they were still connected and didn't try to reconnect, or something like that. In any case, I rebooted them all to get into business again, and now all but the two minis and linux-03 are running. Thanks for looking into this, I hope we get the minis back soon...
Marking the bug 598914 dependency for the issue with Parallels slaves, but that one has been resolved here. What's still open is the minis, and for some unknown reason, cb-sea-miniosx05 is now gone as well. :(
Depends on: 598914
all of these have been rebooted.
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Verified, all 5 minis are back to work! Thanks!
Status: RESOLVED → VERIFIED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.