With this we would like to ease Fedora users to reach our nightly, beta and even stable releases. At first this seems that can be broken down into three subprojects: * Build configuration for the RPM files. * Repository configuration RPMs. * Getting SELinux to work with the nightlies.

Here are the links to the latest Linux binaries for now: * http://people.mozilla.com/~armenzg/latest Once we start getting this into production the slave will actually know the exact URL: * http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/firefox-${version}.en-US.linux-{i686,x86_64}.tar.bz2 * http://releases.mozilla.org/pub/mozilla.org/firefox/releases/${version}/linux-{i686,x86_64}/en-US/firefox-${version}.tar.bz2

There is POC in: http://iraq.proximity.on.ca/ffrepo/ The student project is in: http://zenit.senecac.on.ca/wiki/index.php/Fedora/Firefox_Repository_Setup_for_Nightly_and_Beta_builds I believe the repo creation is explained in here: http://vbluzmans.wordpress.com/2010/11/05/in-quest-of-building-firefoxfedora-repository/ I have to verify that these instructions allow us to install Minefield on Linux: http://tarinc.wordpress.com/2010/11/05/milestone-1-continued/

Here is a blog post explaining how to get updates on Fedora through yum: http://armenzg.blogspot.com/2010/12/getting-firefox-nightly-updates-on.html To install the repo use this rpm: http://chile.proximity.on.ca/ffrepo/noarch/firefox-repo-0.3-2.noarch.rpm For instructions on how to do the setup look at posts by Tarin in December: http://tarinc.wordpress.com/2010/12

Thanks to the work on the previous semester we can now have nightly yum updates for Firefox on the school repositories. For this quarter we have defined to get that work integrated into Mozilla's buildbot automation. I believe is better for me to take on that. If this semester I get more students involved I believe there is room for current students to work on the following: * signing * beta and stable releases * ctyler what else do you think? The only way to know if a new beta is out is to check in here: http://releases.mozilla.org/pub/mozilla.org/firefox/releases Once beta9 is out this week we should have a "4.0b9" directory in there. The only two ways to query what is the next expected directory to appear in there are: A) Check http://mxr.mozilla.org/mozilla-central/source/browser/config/version.txt?raw=1, remove "pre" and substract 1 ("4.0b10pre" - "pre" - 1 => "4.0b9" B) Check in http://releases.mozilla.org/pub/mozilla.org/firefox/releases and increment whatever "4.0b" directory you see there (currently "4.0b8"; we should expect to see "4.0b9" at some point) One day we will have pulse notifications but for now this is what we have. spot has already been doing some work on the betas which can help you leverage: http://repos.fedorapeople.org/repos/spot/firefox4/fedora-14/ The following information will also be valuable to get help and info: General Devel List (devel@fedoraproject.org): https://admin.fedoraproject.org/mailman/listinfo/devel Packaging List (packaging@fedoraproject.org): https://admin.fedoraproject.org/mailman/listinfo/packaging Also, you might try reaching out to the normal firefox maintainers: gecko-maint AT fedoraproject.org Last but not least, the yum maintainer may be able to help you with any yum issues: skvidal

I thought I had attached these files but I guess I didn't. This file gets called everyday and call the other two I have attached. The script gets called twice a day (EST times below) by a cronjob. Once for the Linux 32-bit build and another for the 64-bit one. Tarin I have modified the cronjob to call always with the right version even after it bumps for a beta. I modified the "cut" calls since my URL is a little longer (I think yours was "ftp://ftp.mozilla.org/pub/firefox/" which does not contain "mozilla.org" in it). 0 7 * * * ~/UpdateRepo.bash http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/firefox-`curl -s http://mxr.mozilla.org/mozilla-central/source/browser/config/version.txt?raw=1`.en-US.linux-i686.tar.bz2 0 8 * * * ~/UpdateRepo.bash http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/firefox-`curl -s http://mxr.mozilla.org/mozilla-central/source/browser/config/version.txt?raw=1`.en-US.linux-x86_64.tar.bz2

Due to goals prioritization jhford will be finishing up the work for this quarter. I will debrief him tomorrow.

Currently the repo is down as I can't see anything on here: http://chile.proximity.on.ca/ffrepo/ How can I check why the http server is not running? Because I am lazy I set up an rsync to my people account: rsync -aP armenzg@chile.proximity.on.ca:/var/www/html/ffrepo ~/fedora rsync -aP ~/fedora/ffrepo armenzg@people.mozilla.com:~/public_html This means the repo is now in http://people.mozilla.com/~armenzg/ffrepo/ It might simply not work but I wanted to keep the directory structure of what was available somewhere else. It seems that we have lost the original http://chile.proximity.on.ca/ffrepo/noarch/firefox-repo-0.3-2.noarch.rpm (now http://people.mozilla.com/~armenzg/ffrepo/noarch/). The other rpms are still being generated. I think it would a good opportunity to integrate this system as a mozharness script. This is how the code works. * on chile.p.o.ca the account of Tarin has a cronjob that calls ~/UpdateRepo.bash [1] * UpdateRepo.bash calls ~/includes/BuildSpecFile.bash and then ~/includes/BuildDesktopFile.bash * BuildSpecFile.bash generates ~/rpmbuild/SPECS/minefield.spec * BuildDesktopFile.bash generates ~/rpmbuild/SOURCES/minefield.desktop * At some point we generate the rpm [2] * At some point we generate the srpm [3] * At some point we generate the rpm deltas [4] This page shows how to do updates for the betas and could be helpful for when we tackle adding the betas in our systems: http://www.if-not-true-then-false.com/2010/install-firefox-4-on-fedora/ [1] ################ 0 8 * * * ~/UpdateRepo.bash http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/firefox-`curl -s http://mxr.mozilla.org/mozilla-central/source/browser/config/version.txt?raw=1`.en-US.linux-i686.tar.bz2 0 9 * * * ~/UpdateRepo.bash http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/firefox-`curl -s http://mxr.mozilla.org/mozilla-central/source/browser/config/version.txt?raw=1`.en-US.linux-x86_64.tar.bz2 [2] rpmbuild -ba --quiet --target $arch ~/rpmbuild/SPECS/minefield.spec || exit 1 [3] mv ~/rpmbuild/SRPMS/* $location/$arch/srpms || exit 1 [4] createrepo --update -d --deltas --oldpackagedirs $location/$arch $location/$arch

I am looking into merging in the RPM packaging code into the mozilla build system which makes these patches obsolete. Once I have the code to generate RPMs I am going to create a nightly builder that triggers beside the regular nightly builder. This has to be done in a separate builder because I need to build with mk_add_options MOZ_APP_NAME=minefield to avoid conflicting with the OS version of Firefox.

Diff against 76a6cfcddf47+ tip

Diff against c56c95c4f8c1+ tip

-allows setting per-platform unit test master list -allows setting per-platform toggling of dep builds --caused a massive diff on dep builder, mostly indentation -allows setting per-platform toggling of partials and snippets -allows setting per-platform xulrunner -finds rpms in make upload output -allows setting a different platform than default for things that go to stage, defaults to old platform value -sends debug build (malloc + sdleak) logs to the debug directory -leaves codesize in the opt directory -Uses make vars as needed to generate RPMs -causes mozilla-2.0 platform ordering to change because it no longer copies mozilla-central's platform list When this lands we will need to mv/cp the leak test logs into the correct -debug directory. Maybe a symlink would help here? Also, we should communicate this. This could fix 634968 if we wanted it to

matching config patch

running dump masters + diffs on that then will will ask for review

same patch as before other than the place where the stagePlatform default is set. In v2 it happend before the stripping of 'cruft', which meant that every platform is touched (outside scope of this bug). That might be something worth doing, but later. Moved the default of stagePlatform to after the stripping of 'cruft' to make sure that the current location is what is used.

Same patch as before but instead of enabling nightlies for the platform then disabling on non-mc branches, disable nightlies on all branches that don't explicitly opt into rpm. Kind of a hack, but better than reinventing the wheel. Once rpms become important to people other than me, it might be a good idea to run these rpms on all branches that have nightlies.

diff of a pre and post patch version of the v3 patches

this time, instead of an interdiff between v2 and v3 of the buildbotcustom patch, i am uploading a dump master diff of before and after my patch

changing title to reflect that what was 'beta' is now 'aurora'.

Comment on attachment 525587 [details] [diff] [review] buildbotcustom v3 >diff --git a/misc.py b/misc.py >@@ -1102,6 +1105,9 @@ def generateBranchObjects(config, name): > nightly_builder in l10nNightlyBuilders: > triggeredSchedulers=[l10nNightlyBuilders[nightly_builder]['l10n_builder']] > >+ createSnippet = pf.get('create_snippet', config['create_snippet']) >+ createPartial = pf.get('create_partial', config['create_partial']) >+ > mozilla2_nightly_factory = NightlyBuildFactory( > env=pf['env'], > objdir=pf['platform_objdir'], >@@ -1124,8 +1130,8 @@ def generateBranchObjects(config, name): > uploadPackages=uploadPackages, > uploadSymbols=pf.get('upload_symbols', False), > nightly=True, >- createSnippet=config['create_snippet'], >- createPartial=config['create_partial'], >+ createSnippet=createSnippet, >+ createPartial=createPartial, > ausBaseUploadDir=config['aus2_base_upload_dir'], > updatePlatform=pf['update_platform'], > downloadBaseURL=config['download_base_url'], Is there a reason you don't inline the calls to pf.get('create_*') like we seem to do for the others? >@@ -1149,6 +1155,7 @@ def generateBranchObjects(config, name): > useSharedCheckouts=pf.get('enable_shared_checkouts', False), > testPrettyNames=pf.get('test_pretty_names', False), > l10nCheckTest=pf.get('l10n_check_test', False), >+ stagePlatform=pf.get('stage_platform'), > ) > > mozilla2_nightly_builder = { >@@ -1425,7 +1432,7 @@ def generateBranchObjects(config, name): > blocklistBuilder = generateBlocklistBuilder(config, name, platform, pf['base_name'], pf['slaves']) > branchObjects['builders'].append(blocklistBuilder) > >- if config['enable_xulrunner'] and platform not in ('wince',): >+ if pf.get('enable_xulrunner', config['enable_xulrunner']): > xr_env = pf['env'].copy() > xr_env['SYMBOL_SERVER_USER'] = config['stage_username_xulrunner'] > xr_env['SYMBOL_SERVER_PATH'] = config['symbol_server_xulrunner_path'] Bonus points for removing references to wince! ;) >diff --git a/process/factory.py b/process/factory.py >--- a/process/factory.py >+++ b/process/factory.py >@@ -1,5 +1,6 @@ > from datetime import datetime > import os.path, re >+import posixpath > from time import strftime > import urllib Why do we need posixpath? Does it provide something different than os.path? >+ # it turns out that the cruft is useful sometimes. Instead >+ # of refactoring the entire MercurialBuildFactory and children, >+ # lets create a list of attributes that cause variation. Please be specific as to what the "sometimes" are in the comment, and leave out the bit about refactoring. r+ with comments addressed.

Comment on attachment 525589 [details] [diff] [review] buildbot-configs v3 Just a general comment on this one: As we discussed, having separate platforms for this is OK as an interim step. If we're serious about providing rpms on a long-term basis, rpm creation should be a part of the regular linux build process, and we should be able to toggle it on and off with a single var (or handful of vars). Can we get a bug on file for that improvement, please?

(In reply to comment #21) > Comment on attachment 525587 [details] [diff] [review] > buildbotcustom v3 > Is there a reason you don't inline the calls to pf.get('create_*') like we seem > to do for the others? I had thought we didn't have nested dictionary lookups, but it turns out that we do. Will change. > Bonus points for removing references to wince! ;) :D > Why do we need posixpath? Does it provide something different than os.path? ooops, I thought I had removed that (hg meld sometimes opens temp files instead of real files). I was using posixpath in code i was adding because the os the master runs on is very often unrelated to the code I need to use os.path in. os.path is just posixpath if your interpreter is running on a posix platform. > Please be specific as to what the "sometimes" are in the comment, and leave out > the bit about refactoring. Ok, will do. > r+ with comments addressed. Thanks!

(In reply to comment #22) > Comment on attachment 525589 [details] [diff] [review] > buildbot-configs v3 > > Just a general comment on this one: > > As we discussed, having separate platforms for this is OK as an interim step. > If we're serious about providing rpms on a long-term basis, rpm creation should > be a part of the regular linux build process, and we should be able to toggle > it on and off with a single var (or handful of vars). > > Can we get a bug on file for that improvement, please? Yes, I agree. Until the linux startup process stops using the firefox script, this is a little more difficult. I will file the bug now.

I am going to be working on other projects for the foreseeable future and I won't have time to drive these bugs through to completion. I am marking outstanding packaging fixes as INCOMPLETE.