Closed Bug 806270 Opened 12 years ago Closed 12 years ago

Clean up firefox/candidate-builds/

Categories

(Release Engineering :: General, defect, P2)

x86
All
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: nthomas, Assigned: nthomas)

References

Details

Attachments

(1 file)

Attached file du -sch output (deleted) —
We haven't cleaned up https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/ since moving the stage/ftp hosting to SCL3, and it's consuming 1.5T of space. There are lots of older releases in there that can be trimmed down a lot. For historical purposes it's useful to archive *_info.txt (for buildIDs) logs/* (for archaeology) and we need to keep around update/**/*complete.mar files for any releases we want to generate partials for (or switch to using the firefox/releases/ equivalents). I'll check with the rest of the group if there's anything else to save before we delete the rest (most of which is duplicated in firefox/releases/ anyway).
Update verify already pulls old installers from firefox/releases/.
Hi Nick, do you have a timeline for trimming this down?
I've been creating a script to do the cleanup but hit a snag with the contributed Solaris builds. Those are uploaded with the account dave.lin, although I believe someone else at Oracle now actually does the uploading now, and end up with permissions which mean ffxbld can't delete them. Any thoughts on how we might work around that, given no-one has sudo these days ? Perhaps we can change the umask on that account, or use cron to recursively chmod files in /pub/mozilla.org/{firefox,thunderbird}/candidates/*/*/contrib/.
Or we could delete this one for you, assuming new uploads have the right perms.
They don't, so we should set the umask and get chmod run. I'll open a bug for that.
Depends on: 818758
Currently 1.2T free on this share, and waiting for bug 818759. Will revisit this early in 2013.
> Currently 1.2T free on this share, and waiting for bug 818759. Yeah, I bumped tinderbox_builds up on 2012-12-17 and rebalanced snap reserve to cover the volume as it stands these days. > Will revisit this early in 2013. Yes, please. :)
gcox/cshields: Trimmed it down usage to 525G, probably would have been over 2T by now. I suspect the savings won't show up for a week, when the netapp expires the weekly snapshot and actually deletes the data of disk. I've needed to leave some versions around until bug 839926 is done. Bug 748811 tracks an automated fix for cleaning up. RelEng: The script for this is on bug 748811. I got carried away and deleted firefox/candidates/10.0.12esr-candidates/ but we'll still need that for updates to 17 ESR. Thankfully this NFS partition has the NetApp snapshot feature still enabled, so I was able to restore the files from /mnt/cm-ixstore01/.snapshot/hourly.1/... with an rsync -av, so it should be back as it was. (Just FYI if something blows up later)
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Oh, and the script archives only the buildbot logs, which can be found in http://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/archived/ 70 releases taking up 3G of space (logs are already gzipped).
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: