Closed
Bug 1415268
Opened 7 years ago
Closed 7 years ago
Internal Server Error while trying to set up an ESR build
Categories
(Release Engineering :: Applications: Shipit, defect)
Release Engineering
Applications: Shipit
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: lizzard, Assigned: jlorenzo)
References
Details
Attachments
(2 files)
I tried twice to set up a build for ESR 52.5.0 (build 2) but got an internal server error.
Here's the error message: "The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application."
Reporter | ||
Comment 1•7 years ago
|
||
jlorenzo pointed out this is from a very long list of suggested partial builds.
Assignee | ||
Comment 2•7 years ago
|
||
:jcristau and I investigated this issue earlier today. Leaving a NI to myself so I write down the results and I track this issue for ESR 52.6.0 (in 6 weeks)
Flags: needinfo?(jlorenzo)
Assignee | ||
Comment 3•7 years ago
|
||
There are 2 issues here:
1. The server should refuse input that contain more than X partials. X to be determined.
2. The string comparison done at [1] is not valid anymore. It compares 52.1.0esr to 52.1.0. I don't understand how the behavior has changed since ESR 52.4.0.
Marking it as blocking the next merge day (bug 1412962).
[1] https://github.com/mozilla-releng/ship-it/blob/23042ec72ccf4242e31545b99800ca311c2af93a/kickoff/static/suggestions.js#L120
Blocks: 1412962
Flags: needinfo?(jlorenzo)
Assignee | ||
Comment 5•7 years ago
|
||
Fixes issue #2.
Assignee: nobody → jlorenzo
Attachment #8929492 -
Flags: review?(nthomas)
Assignee | ||
Comment 6•7 years ago
|
||
(In reply to Johan Lorenzo [:jlorenzo] from comment #3)
> I don't understand how the behavior has changed since ESR 52.4.0.
I now do \o/. TL;DR: That's because we shipped 52.4.1 to MacOS users only.
All partials used here[1] are taken from partial.json[2]. This file is overwritten from times to times. Most importantly, it's ordered by number of ADIs. We shipped 52.4.1 to Mac users only[3], leaving the rest of the population on 52.4.0. Hence, partial.json was ordered this way:
> {
> "esr": [{
> "version": "52.4.0esr",
> "ADI": 40000
> }, {
> "version": "52.3.0esr",
> "ADI": 30000
> }, {
> "version": "52.4.1esr",
> "ADI": 20000
> }]
> }
(Don't mind the numbers, I replaced them by dummy ones)
So populatePartial() first added 52.4.1 because it's the last shipped release[4], then added the topmost release in partial.json (which was 52.4.0), but couldn't exit the loop, because of this strict equal condition.
Attachment 8929492 [details] will fix this. I added a new test scenario [7] to validate the theory and the fix.
[1] https://github.com/mozilla-releng/ship-it/blob/9c6ecbac45fe7f5e8f1a4f5bbde43361954a3847/kickoff/static/suggestions.js#L174
[2] https://ship-it.mozilla.org/cron/partial.json
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=1404481#c7
[4] https://github.com/mozilla-releng/ship-it/blob/9c6ecbac45fe7f5e8f1a4f5bbde43361954a3847/kickoff/static/suggestions.js#L205
[5] https://github.com/mozilla-releng/ship-it/blob/9c6ecbac45fe7f5e8f1a4f5bbde43361954a3847/kickoff/static/suggestions.js#L228
[6] https://github.com/mozilla-releng/ship-it/blob/9c6ecbac45fe7f5e8f1a4f5bbde43361954a3847/kickoff/static/suggestions.js#L232
[7] https://github.com/mozilla-releng/ship-it/pull/207/commits/1dc08a10be6a018819927c22dec2167a489dc1aa
Updated•7 years ago
|
Attachment #8929490 -
Flags: review?(rail) → review+
Assignee | ||
Comment 7•7 years ago
|
||
Comment on attachment 8929492 [details]
Ship-it PR - Prevent front end to add to many partials
Was r+'d at https://github.com/mozilla-releng/ship-it/pull/207#pullrequestreview-77993670.
Attachment #8929492 -
Flags: review?(nthomas) → review+
Assignee | ||
Comment 8•7 years ago
|
||
Landed both fixes at:
* https://github.com/mozilla-releng/ship-it/commit/18dc35c511c2d38a1f7472c34c435176f3807212
* https://github.com/mozilla-releng/ship-it/commit/c62574340c1688f659dadc22f9b0a73beaa35854
I deployed and manually tested them to prod. Crafting 52.5.0 (build3) was still an issue in prod, and it's no more.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Updated•3 years ago
|
Component: Applications: ShipIt (backend) → Applications: ShipIt
You need to log in
before you can comment on or make changes to this bug.
Description
•