Closed Bug 1148592 Opened 10 years ago Closed 9 years ago

figure how to accept contrib builds in s3

Categories

(Release Engineering :: Release Automation: Other, defect)

x86_64
Linux
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: bhearsum, Unassigned)

References

Details

...or decide not to host them ourselves anymore. We've long accepted contrib builds for tier-2 and below platforms. For the past few years, this has primarily been Solaris builds. It looks like they still try to make them for ESR. Eg: http://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/31.5.0esr-candidates/build2/contrib/solaris_pkgadd/ They way they're delivered to us right now is by uploading them to stage.mozilla.org over ssh. The "contrib" directories have wide enough permissions that certain users are allowed to put files in them. Perhaps we can give the appropriate people write (but not overwrite) ability to certain buckets in s3? Or use a separate bucket for contrib builds? I don't see links from our website to the Solaris builds, so I'm not sure it matters much where they are as long as they're accessible.
AFAIK we're just hosting them on behalf of contributors, who get a bit of reputation rubbing off on them in the process.
In a meeting today we chewed on the idea of using a separate bucket for contrib builds, as that makes the auth policy for candidates and releases dirs much simpler. It didn't sound like we could map that back to the current layout when serving with the CDN, but it doesn't seem that big a deal to ask people to use a different url to distribute. If that's wrong we can automate dropping a README or a 302 for each new release.
Blocks: 1165405
No longer blocks: 1147140
We agreed on a follow on meeting to brainstorm a solution. Attendees: nthomas, oremj, tblow. Who else do we need? bhearsum? catlee? others?
We met today and agreed to investigate a separate bucket, with a redirect to preserve the existing 'discoverability' of the current location (for people browsing the directories). eg something like firefox_bucket:/releases/<version>/contrib/ --> contrib_bucket:/firefox/releases/<version>/ for once we're shipped. Not yet determined what we'll do about candidates. We'll try to trim down the list of accounts with contrib access, and disable accounts that haven't been used in some number of months. Something will need to create the directories in the contrib bucket, and the redirects in the firefox bucket. The candidates part could be handled by post_upload.py, at the point it creates a candidates directory; while the releases part could be done by the reinvented push_to_mirrors operation. nthomas & oremj to work on this over the next 1-2 weeks.
Is this resolved? If not, what additional work needs to be done?
I've checked into S3 redirects, and you can set metdata called x-amz-website-redirect-location on a key, which specifies a relative or absolute url. So I just need to know if we're going to have https://archive.mozilla.org/pub/<product>/releases/<version>/contrib/ which I can point to https://archive.mozilla.org/pub/contrib/<product>/<version>/ using '../../../../contrib/<product>/<version>/'. If use a relative url like that then the specifics of the domain obviously don't matter, just the layout of paths.
Flags: needinfo?(oremj)
Oops, forgot about handling multiple builds before release. Something like this instead: In pushed we release we'd have https://archive.mozilla.org/pub/<product>/releases/<version>/contrib/ which I can point to https://archive.mozilla.org/pub/contrib/<product>/<version>/build<N>/ using '../../../../contrib/<product>/<version>/build<N>/'. If we use a relative url like that then the specifics of the domain obviously don't matter, just the layout of paths.
That looks good. If we want https://archive.mozilla.org/pub/contrib/<product>/<version>/build<N>/ to be 200 OK, we'll need to upload some file, a README perhaps, there on release as well. Also, I've never tested relative redirects, so we need to make sure that works.
Flags: needinfo?(oremj)
I believe we solved this in bug 1189607 by granting access directly at the usual location.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.