Closed Bug 1415268 Opened 7 years ago Closed 7 years ago

Internal Server Error while trying to set up an ESR build

Categories

(Release Engineering :: Applications: Shipit, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: lizzard, Assigned: jlorenzo)

References

Details

Attachments

(2 files)

I tried twice to set up a build for ESR 52.5.0 (build 2) but got an internal server error. Here's the error message: "The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application."
jlorenzo pointed out this is from a very long list of suggested partial builds.
:jcristau and I investigated this issue earlier today. Leaving a NI to myself so I write down the results and I track this issue for ESR 52.6.0 (in 6 weeks)
Flags: needinfo?(jlorenzo)
There are 2 issues here: 1. The server should refuse input that contain more than X partials. X to be determined. 2. The string comparison done at [1] is not valid anymore. It compares 52.1.0esr to 52.1.0. I don't understand how the behavior has changed since ESR 52.4.0. Marking it as blocking the next merge day (bug 1412962). [1] https://github.com/mozilla-releng/ship-it/blob/23042ec72ccf4242e31545b99800ca311c2af93a/kickoff/static/suggestions.js#L120
Blocks: 1412962
Flags: needinfo?(jlorenzo)
Fixes issue #2.
Assignee: nobody → jlorenzo
Attachment #8929492 - Flags: review?(nthomas)
(In reply to Johan Lorenzo [:jlorenzo] from comment #3) > I don't understand how the behavior has changed since ESR 52.4.0. I now do \o/. TL;DR: That's because we shipped 52.4.1 to MacOS users only. All partials used here[1] are taken from partial.json[2]. This file is overwritten from times to times. Most importantly, it's ordered by number of ADIs. We shipped 52.4.1 to Mac users only[3], leaving the rest of the population on 52.4.0. Hence, partial.json was ordered this way: > { > "esr": [{ > "version": "52.4.0esr", > "ADI": 40000 > }, { > "version": "52.3.0esr", > "ADI": 30000 > }, { > "version": "52.4.1esr", > "ADI": 20000 > }] > } (Don't mind the numbers, I replaced them by dummy ones) So populatePartial() first added 52.4.1 because it's the last shipped release[4], then added the topmost release in partial.json (which was 52.4.0), but couldn't exit the loop, because of this strict equal condition. Attachment 8929492 [details] will fix this. I added a new test scenario [7] to validate the theory and the fix. [1] https://github.com/mozilla-releng/ship-it/blob/9c6ecbac45fe7f5e8f1a4f5bbde43361954a3847/kickoff/static/suggestions.js#L174 [2] https://ship-it.mozilla.org/cron/partial.json [3] https://bugzilla.mozilla.org/show_bug.cgi?id=1404481#c7 [4] https://github.com/mozilla-releng/ship-it/blob/9c6ecbac45fe7f5e8f1a4f5bbde43361954a3847/kickoff/static/suggestions.js#L205 [5] https://github.com/mozilla-releng/ship-it/blob/9c6ecbac45fe7f5e8f1a4f5bbde43361954a3847/kickoff/static/suggestions.js#L228 [6] https://github.com/mozilla-releng/ship-it/blob/9c6ecbac45fe7f5e8f1a4f5bbde43361954a3847/kickoff/static/suggestions.js#L232 [7] https://github.com/mozilla-releng/ship-it/pull/207/commits/1dc08a10be6a018819927c22dec2167a489dc1aa
Attachment #8929490 - Flags: review?(rail) → review+
Comment on attachment 8929492 [details] Ship-it PR - Prevent front end to add to many partials Was r+'d at https://github.com/mozilla-releng/ship-it/pull/207#pullrequestreview-77993670.
Attachment #8929492 - Flags: review?(nthomas) → review+
Landed both fixes at: * https://github.com/mozilla-releng/ship-it/commit/18dc35c511c2d38a1f7472c34c435176f3807212 * https://github.com/mozilla-releng/ship-it/commit/c62574340c1688f659dadc22f9b0a73beaa35854 I deployed and manually tested them to prod. Crafting 52.5.0 (build3) was still an issue in prod, and it's no more.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
No longer blocks: 1575613
Component: Applications: ShipIt (backend) → Applications: ShipIt
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: