To export the current list of releases and history, we used a series of PHP which will export the data into json files. I think we should use the ship-it database to replace the old PHP(4) library. For those who are not familiar with the product details, we have a few PHP files which we update by hand (should be automatic soon, see bug 1053814). Basically, we provide: * current aurora, beta & release version * history of these versions This applies for Firefox, mobile & Thunderbird (but not Firefox OS). We used to include PHP files when our websites were developed in PHP. I started to hack on this to remove the deprecated code and improve the json schema (bug 1080532). However, the www team is complaining that I am breaking many things (even if the values are not being used... like the filesize). So, instead of hacking on top of product-details to add a new schema (and adding more legacy management on top of the product detail), I would like to know if it could make sense to use ship-it to manage that. I see a few issues to make that happens but I don't see any blocker. * it is only on the VPN while the json needs to be exposed publicly. We could just use the ship-it database (without its front end) * we don't have the locale info. Example: var $languages = array( 'af' => array( 'English' => 'Afrikaans', 'native' => 'Afrikaans'), and the properties files * we don't have the whole history but that should be trivial to import. * we don't know yet if build has been shipped or not (but this is on the releng roadmap)

Here is a proof of concept. It is still need: * thunderbird json files * mobile_details.json to be implemented [1] * import of the history * cleaning [1] The schema seems a bit odd to me: { "version": "33.0", "beta_version": "34.0b4", "alpha_version": "35.0a2", "builds": [ { "locale": { "code": "ar", "english": "Arabic", "native": "\u0639\u0631\u0628\u064a" } }, { "locale": { [...] "beta_builds": [ { "locale": { "code": "cs", "english": "Czech", "native": "\u010ce\u0161tina" }, "download": { "android": "market:\/\/details?id=org.mozilla.firefox_beta" } }, { "locale": { "code": "de", "english": "German", [...] ], "alpha_builds": [ { "locale": { "code": "en-US", "english": "English (US)", "native": "English (US)" }, "download": { "android": "market:\/\/details?id=org.mozilla.firefox" } } ] }

Not ready but getting closer.

FYI, I am now working on this branch: https://github.com/sylvestre/ship-it/tree/json and travis: https://travis-ci.org/sylvestre/ship-it/builds

Mass component change for ship it bugs.

Any ETA here? I'd like to have Thunderbird data for Bug 1123399, but if this takes time, I may probably go with the legacy product-details instead.

The goal is to provide the exact same json files as in p-d. So, you will/should have to update only the URL. Anyway, I just got a VM to finalize the work. Hopefully, it is going to be live in April.

Oh, okay. Looks like I was confused. Thunderbird stable version data is already available in p-d. I wanted to see the Thunderbird Beta & Earlybird data too, but it can be added later. Thanks.

So, with my tb hat (I have been helping lately), it is something that we are not updating but we can start doing it if needed.

I know the Thunderbird Beta info is available in http://viewvc.svn.mozilla.org/vc/projects/mozilla.org/branches/staging/thunderbird/includes/thunderbirdBetaBuildDetails.php It would be appreciated if this info could be added to the main p-d lib.

The patch from Github has been merged. We're planning to get it landed in production today, but git.m.o is having some issues at the moment (bug 1176192). If those are sorted out early enough my plan is to get this + bug 1149978 pushed to ship-it-dev to do some verification, and then get that pushed to prod.

This is in dev, but broken because of this ISE 500: 2015-06-19 14:52:00,833 - kickoff.log_exception#1306: Exception on /releases.html [GET] Traceback (most recent call last): File "/data/www/ship-it-dev.allizom.org/release-kickoff/vendor/lib/python/flask/app.py", line 1687, in wsgi_app response = self.full_dispatch_request() File "/data/www/ship-it-dev.allizom.org/release-kickoff/vendor/lib/python/flask/app.py", line 1360, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/lib64/python2.6/site-packages/newrelic-2.32.0.28/newrelic/hooks/framework_flask.py", line 98, in _nr_wrapper_Flask_handle_exception_ return wrapped(*args, **kwargs) File "/data/www/ship-it-dev.allizom.org/release-kickoff/vendor/lib/python/flask/app.py", line 1358, in full_dispatch_request rv = self.dispatch_request() File "/data/www/ship-it-dev.allizom.org/release-kickoff/vendor/lib/python/flask/app.py", line 1344, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/usr/lib64/python2.6/site-packages/newrelic-2.32.0.28/newrelic/hooks/framework_flask.py", line 40, in _nr_wrapper_handler_ return wrapped(*args, **kwargs) File "/data/www/ship-it-dev.allizom.org/release-kickoff/vendor/lib/python/flask/views.py", line 83, in view return self.dispatch_request(*args, **kwargs) File "/data/www/ship-it-dev.allizom.org/release-kickoff/vendor/lib/python/flask/views.py", line 150, in dispatch_request return meth(*args, **kwargs) File "/data/www/ship-it-dev.allizom.org/release-kickoff/kickoff/views/releases.py", line 113, in get return render_template('releases.html', releases=sortedReleases(), form=form) File "/data/www/ship-it-dev.allizom.org/release-kickoff/kickoff/views/releases.py", line 32, in sortedReleases return sorted(getReleases(), cmp=cmpReleases) File "/data/www/ship-it-dev.allizom.org/release-kickoff/kickoff/views/releases.py", line 31, in cmpReleases return cmp(y._submittedAt, x._submittedAt) TypeError: can't compare datetime.datetime to NoneType

The previous error was fixed by making sure submittedAt was set for all rows (more fallout from bug 1149978). Sylvestre has identified two more issues: 1) https://ship-it-dev.allizom.org/json/firefox_primary_builds.json => error 500 2015-06-19 17:02:15,834 - kickoff.log_exception#1306: Exception on /json/firefox_primary_builds.json [GET] Traceback (most recent call last): File "/data/www/ship-it-dev.allizom.org/release-kickoff/vendor/lib/python/flask/app.py", line 1687, in wsgi_app response = self.full_dispatch_request() File "/data/www/ship-it-dev.allizom.org/release-kickoff/vendor/lib/python/flask/app.py", line 1360, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/lib64/python2.6/site-packages/newrelic-2.32.0.28/newrelic/hooks/framework_flask.py", line 98, in _nr_wrapper_Flask_handle_exception_ return wrapped(*args, **kwargs) File "/data/www/ship-it-dev.allizom.org/release-kickoff/vendor/lib/python/flask/app.py", line 1358, in full_dispatch_request rv = self.dispatch_request() File "/data/www/ship-it-dev.allizom.org/release-kickoff/vendor/lib/python/flask/app.py", line 1344, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/usr/lib64/python2.6/site-packages/newrelic-2.32.0.28/newrelic/hooks/framework_flask.py", line 40, in _nr_wrapper_handler_ return wrapped(*args, **kwargs) File "/data/www/ship-it-dev.allizom.org/release-kickoff/kickoff/jsonexport.py", line 155, in firefox_primary_builds_json buildsVersionLocales = updateLocaleWithVersionsTable("firefox") File "/data/www/ship-it-dev.allizom.org/release-kickoff/kickoff/jsonexport.py", line 136, in updateLocaleWithVersionsTable lastStable[0][0]) File "/data/www/ship-it-dev.allizom.org/release-kickoff/kickoff/jsonexport.py", line 109, in generateLocalizedBuilds locales = parsePlainL10nChangesets(l10nchangesets) File "/data/www/ship-it-dev.allizom.org/release-kickoff/vendor/lib/python/mozilla/release/l10n.py", line 4, in parsePlainL10nChangesets locale, revision = line.rstrip().split() ValueError: need more than 1 value to unpack 2) https://ship-it-dev.allizom.org/json/languages.json => not found However, he thinks this doesn't block going to production. If today's Beta ships in the next couple of hours I'll go ahead and push to prod. If not, this waits until next week.

This has been pushed to prod and I _think_ it's looking fine. I also did the necessary imports from bug 1149978. Sylvestre, can you verify?

When is ETA for ship-it to replace https://svn.mozilla.org/libs/product-details/json/ JSONs? Is there any wikipage to track the current development?

I think this bug can be finally closed. For more information: see bug 1288169 and bug 1288171