compare-locales.pl has bugs, and it's mangling build logic with application rules like exceptions for region.properties. It's hardly maintained, and hard to do. testing/tests/l10n's compare-locales is nicer, we should use that instead. I filed bug 400009 to check that out by default, so that tinderbox could then use it.

Hey Axel, are you planning to do this? If not, can we close this out WONTFIX?

compare-locale to have it installed into all slaves with easy install? or have a step to check it out every time and then run it accordingly?

We should be checking it out every time, afaict.

Should this block on getting the changes from http://hg.mozilla.org/users/axel_mozilla.com/tooling/index.cgi/file/tip/mozilla/testing/tests/l10n/ reviewed and merged back into cvs? Independent of this is the current code (both in cvs and hg) assuming that you ran python setup.py install. I think that's generally the right idea, but may not be for tinderbox. We might have to hack some PYTHONPATH trickeries there.

(In reply to comment #4) > Should this block on getting the changes from > http://hg.mozilla.org/users/axel_mozilla.com/tooling/index.cgi/file/tip/mozilla/testing/tests/l10n/ > reviewed and merged back into cvs? > > Independent of this is the current code (both in cvs and hg) assuming that you > ran python setup.py install. I think that's generally the right idea, but may > not be for tinderbox. I would prefer that we be checking out the code each time and specifically not install into site-packages. This greatly reduces the PITA factor when making changes to it.

There shouldn't be changes to it *that* often. I'd like to point out that all the application logic is not part of compare-locales anymore, i.e., which directories to compare and which files and entities to ignore are pulled from the actual application source. Specifically, the dirs are extracted from client.mk via make -f client.mk echo-variable-LOCALES_browser and the exception rules are coded in browser/locales/filter.py. I'm not that worried about the tinderbox setup, but when moving this to the buildbot world, we shouldn't let deployment problems determine how efficient we get. Not that I think that the deployment problems will be as bad as they're now then, with redundant slaves and all that.

How does installing into site-packages effect efficiency? Who is the lucky one that has to go around and update each slave when a change does happen?

Let's move that discussion out of this bug, because for tinderbox, it really doesn't matter? It will matter for the buildbot scheme, which is in sketched out in bug 393127. Short story: To get the information I need for release automation up to shipped-locales, the most efficient way I found is to run compare-locales as proper python within the slave. Where "release automation" is me re-using that word for the semi-automated process to generate shipped-locales. Filed bug 436488 for the bigger picture.

> It will matter for the buildbot scheme, which is in sketched out in bug 393127. > Short story: To get the information I need for release automation up to > shipped-locales, the most efficient way I found is to run compare-locales as > proper python within the slave. Where "release automation" is me re-using that > word for the semi-automated process to generate shipped-locales. Filed bug > 436488 for the bigger picture. > I can't tell you what to do with your own Buildbots, but I do not want releng-supported ones to need manual poking if/when compare-locales changes. To be clear: I'm not suggesting compare-locales become a BuildStep, or something, I'm simply talking about either a) FileDownload'ing it to the slave, or b) checking it out from CVS/hg on every run. Either way it can be run with "python proper".

To be clear, I'm talking about in-slave-process run python classes implementing ISlaveCommand, the stuff you just found too hard to fix in http://buildbot.net/trac/ticket/221#comment:5.

(In reply to comment #10) > To be clear, I'm talking about in-slave-process run python classes implementing > ISlaveCommand, the stuff you just found too hard to fix in > http://buildbot.net/trac/ticket/221#comment:5. > That ticket is about master-side classes being dynamically transmitted to a slave - not pre-installed slave-side code. Independent of slave-side Buildbot code, is there a way to run all of this _without_ Buildbot?

(In reply to comment #11) > Independent of slave-side Buildbot code, is there a way to run all of this > _without_ Buildbot? Yes and no. You can run the functionally equivalent code for the actual comparison as a regular python-based script, http://developer.mozilla.org/en/docs/Compare-locales. The output it produces is adapted to terminals. Better than what compare-locales.pl does, but yeah. The pieces that don't work standalone are the integration and data mining pieces on the master. They master is getting the raw data from the slave as a python dict, stores that in a db, and in a build property. Various status plugins post-process that data, to get aggregatable results used on http://l10n.mozilla.org/dashboard/, and visually easy comparison outputs like http://l10n.mozilla.org/buildbot/compare/linux-langpack/11146. It provides some evolutional data, too, see http://l10n.mozilla.org/buildbot/statistics?buildername=linux-langpack&tree=trunk&app=browser&locale=si.

I have raised the priority of this bug and say to make "buildbot" to call the right compare-locale instead of "tinderbox" since the l10n repackages is now buildbot driven

I have filed bug 458938 to coordinate this bug and bug 458152 As per conversation with Axel this should not be blocking bug 458153

John to talk with Armen and Axel to figure out if this is a dup of one of many other l10n bugs?

the cvs version is dead meat, resolving WONTFIX.