/home/cltbld/build/maemo-trunk-nightly/mozilla-central/objdir/config/nsinstall -D ./dist/crashreporter-symbols /usr/bin/python2.5 /home/cltbld/build/maemo-trunk-nightly/mozilla-central/toolkit/crashreporter/tools/symbolstore.py \ -c --vcs-info \ -s /home/cltbld/build/maemo-trunk-nightly/mozilla-central \ ./dist/host/bin/dump_syms \ ./dist/crashreporter-symbols \ ./dist/bin > \ ./dist/crashreporter-symbols/fennec-2.0a1pre-Linux-20100523014056-symbols.txt Processing file: ./dist/bin/TestDNS Unexpected error: <type 'exceptions.OSError'> Traceback (most recent call last): File "/home/cltbld/build/maemo-trunk-nightly/mozilla-central/toolkit/crashreporter/tools/symbolstore.py", line 784, in <module> main() File "/home/cltbld/build/maemo-trunk-nightly/mozilla-central/toolkit/crashreporter/tools/symbolstore.py", line 780, in main dumper.Process(arg) File "/home/cltbld/build/maemo-trunk-nightly/mozilla-central/toolkit/crashreporter/tools/symbolstore.py", line 466, in Process return self.ProcessDir(file_or_dir) File "/home/cltbld/build/maemo-trunk-nightly/mozilla-central/toolkit/crashreporter/tools/symbolstore.py", line 483, in ProcessDir if not self.ProcessFile(fullpath): File "/home/cltbld/build/maemo-trunk-nightly/mozilla-central/toolkit/crashreporter/tools/symbolstore.py", line 531, in ProcessFile (filename, rootname) = GetVCSFilename(filename, self.srcdirs) File "/home/cltbld/build/maemo-trunk-nightly/mozilla-central/toolkit/crashreporter/tools/symbolstore.py", line 365, in GetVCSFilename fileInfo = HGFileInfo(file, srcdir) File "/home/cltbld/build/maemo-trunk-nightly/mozilla-central/toolkit/crashreporter/tools/symbolstore.py", line 281, in __init__ 'parent', '--template={node|short}') File "/home/cltbld/build/maemo-trunk-nightly/mozilla-central/toolkit/crashreporter/tools/symbolstore.py", line 263, in read_output (stdout, _) = Popen(args=args, stdout=PIPE).communicate() File "/usr/lib/python2.5/subprocess.py", line 594, in __init__ errread, errwrite) File "/usr/lib/python2.5/subprocess.py", line 1086, in _execute_child data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB OSError: [Errno 4] Interrupted system call make: *** [buildsymbols] Error 1 qemu: uncaught target signal 1 (Hangup) - exiting jhford -- how do we leverage the host-native python in maemo4 scratchbox land?

we need to remove python2.5, python2.5-dev and all python tools (including HG) and run the python part of https://wiki.mozilla.org/ReferencePlatforms/Linux-CentOS-5.0#Upgrade_Scratchbox I can take this

going to find out which files are modified by running find /builds/scratchbox/ -type f -exec openssl md5 "{}" \; -o -type d -exec echo DIR\("{}"\) \; before and after changes

Hit this during 1.1rc1 build1. Going to try to force it to build on a VM.

i am going to deploy this as a host rpm file that will add files to the scratchbox chinook sdk target. I am going to build the python interpreter outside of the RPM script as I'd rather not deal with scratchbox building inside of an rpmspec file. Before each installation of this package, we are going to need to run /builds/scratchbox/moz_scratchbox sb-conf select CHINOOK-ARMEL-2007 /builds/scratchbox/moz_scratchbox fakeroot apt-get remove python2.5 -y Doing this removal of python2.5 results in the following packages being removed as well: python2.5-xml python2.5-dev python2.5-distutils I am not sure if HG is going to work in the chinook sdk after this upgrade, but that is one of the things that I plan to test

This is what I have so far: Before you install host-python for maemo4, run this /builds/scratchbox/moz_scratchbox sb-conf select CHINOOK-ARMEL-2007 /builds/scratchbox/moz_scratchbox fakeroot apt-get remove python2.5 -y Which will remove these packages python-dev python-xml python-distutils You will also need to remove the PYTHONPATH= line in ~/.bashrc These are the steps to build x86 python for maemo4: /builds/scratchbox/moz_scratchbox wget http://www.python.org/ftp/python/2.5.5/Python-2.5.5.tar.bz2 wget http://www.openssl.org/source/openssl-0.9.8m.tar.gz wget http://mercurial.selenic.com/release/mercurial-1.5.1.tar.gz wget http://www.bzip.org/1.0.5/bzip2-1.0.5.tar.gz tar jxf Python-2.5.5.tar.bz2 tar zxf openssl-0.9.8m.tar.gz tar zxf mercurial-1.5.1.tar.gz tar zxf bzip-1.0.5.tar.gz cd openssl-0.9.8m CC=host-gcc CXX=host-g++ ./config shared --prefix=/usr/local make -j4 make install cd ../bzip-1.0.5 make CC=host-gcc PREFIX=/usr/local install make CC=host-gcc -f Makefile-libbz2_so cp libbz2.so.1.0.4 /usr/local/lib/ cd ../Python-2.5.5 CC=host-gcc CXX=host-g++ ./configure --prefix=/usr/local make -j4 make altinstall cd ../mercurial-1.5.1 python2.5 setup.py install find /usr/local -name "*.so*" -exec ldd {} \; | grep "not found" # should have no output Because of how the PATH var is set, and our build system's requirement of having the mercurial executable be callable with 'hg', it seems like bug 578906 should block this work.

(In reply to comment #5) > Because of how the PATH var is set, and our build system's requirement of > having the mercurial executable be callable with 'hg', it seems like bug 578906 > should block this work. to be more specific, the work around I have right now is to use a wrapper script 'hg' that calls the real mercurial script the right way. The problem is that this requires putting the wrapper script in a place that is also what the FREMANTLE target uses. Changing the python interpreter to be the same interpreter (in /host_usr) for both fremantle and chinook and have all of this work together seems outside of the scope of this bug. My plan is to see how difficult bug 578906 will be to fix before I invest the time to make the scratchbox work properly with a shared python interpreter and application stack. Ben, for the deployment of whatever we go with, I am leaning towards creating a new RPM for the entire scratchbox. The alternative is to have an rpm that does lots of things in its %preinst and %postinst that are dodgy at best (including removing a package inside of the scratchbox). Will the proxy help us with MPT->MV load if we do go with this deployment? Do we know that rpm will fail to install a corrupt package and block the puppet run if there is some cache corruption?

(In reply to comment #6) > Ben, for the deployment of whatever we go with, I am leaning towards creating a > new RPM for the entire scratchbox. The alternative is to have an rpm that does > lots of things in its %preinst and %postinst that are dodgy at best (including > removing a package inside of the scratchbox). Will the proxy help us with > MPT->MV load if we do go with this deployment? Do we know that rpm will fail > to install a corrupt package and block the puppet run if there is some cache > corruption? The proxy will mostly make the load issues moot, but I really don't want to set a precedent here. Even if its a little bit ugly, I think it's cleaner to create an incremental package than redeploy the whole thing. Not to mention: quicker. Can you spell out what needs to be done %preinst and %postinst? Things like package removal may be able to be done in Puppet rather than %preinst.

(In reply to comment #7) > The proxy will mostly make the load issues moot, but I really don't want to set > a precedent here. Even if its a little bit ugly, I think it's cleaner to create > an incremental package than redeploy the whole thing. Not to mention: quicker. > > Can you spell out what needs to be done %preinst and %postinst? Things like > package removal may be able to be done in Puppet rather than %preinst. it is going to be at a minimum: %preinst /builds/scratchbox/sbin/sbox_ctl stop /builds/scratchbox/sbin/sbox_umount_all /builds/scratchbox/moz_scratchbox sb-conf select CHINOOK-ARMEL-2007 /builds/scratchbox/moz_scratchbox fakeroot apt-get remove python2.5 -y %postinst /builds/scratchbox/sbin/sbox_ctl start Having it as a file delta is going to be a ton of extra work to generate because of what I am changing. It is going to break the scratchbox-mercurial package that is already there (which we could remove before the install). This will also install files that are going to cross targets which makes it very difficult to build the rpm incrementally. I am also unsure what issues there will be if we bring up a new slave. I understand that we don't want to set a precedent for a single monolithic rpm for all changes, but i really do feel like this case it is the correct way to deploy. For changes that are only going to add files to the scratchbox filesystem, I think doing extra rpms is the best way to go but in this case, we need to change too many things to make this feasible. I also don't feel comfortable that we will end up with a working scratchbox at the end of this process. If we have infrastructure to deal with large packages without murdering our network, I think we should use that instead of doing a risky incremental package that is likely to be broken and take much longer to create. Is puppet using the squid cache or a separate proxy?

(In reply to comment #8) > it is going to be at a minimum: > > %preinst > /builds/scratchbox/sbin/sbox_ctl stop > /builds/scratchbox/sbin/sbox_umount_all > /builds/scratchbox/moz_scratchbox sb-conf select CHINOOK-ARMEL-2007 > /builds/scratchbox/moz_scratchbox fakeroot apt-get remove python2.5 -y > > > %postinst > /builds/scratchbox/sbin/sbox_ctl start This all seems perfectly reasonable to do. > It is going to break the scratchbox-mercurial > package that is already there (which we could remove before the install). This is a non-issue, then. > This > will also install files that are going to cross targets which makes it very > difficult to build the rpm incrementally. Why does that make it difficult? Based on earlier comments, you wouldn't be working inside of Scratchbox in rpmbuild, so I wouldn't think crossing targets would be an issue. > I am also unsure what issues there > will be if we bring up a new slave. How would bringing up a new slave cause different issues than upgrading a current one? > I understand that we don't want to set a precedent for a single monolithic rpm > for all changes, but i really do feel like this case it is the correct way to > deploy. For changes that are only going to add files to the scratchbox > filesystem, I think doing extra rpms is the best way to go but in this case, we > need to change too many things to make this feasible. I also don't feel > comfortable that we will end up with a working scratchbox at the end of this > process. If we have infrastructure to deal with large packages without > murdering our network, I think we should use that instead of doing a risky > incremental package that is likely to be broken and take much longer to create. Why don't you feel comfortable that we'll have a working Scratchbox? I don't expect we'd roll anything out that didn't first test OK. I don't think testing and verifying workness is complicated here. > Is puppet using the squid cache or a separate proxy? squid

(In reply to comment #9) > > This > > will also install files that are going to cross targets which makes it very > > difficult to build the rpm incrementally. > > Why does that make it difficult? Based on earlier comments, you wouldn't be > working inside of Scratchbox in rpmbuild, so I wouldn't think crossing targets > would be an issue. Generating the list of files to include in the rpm is going to be very difficult. > Why don't you feel comfortable that we'll have a working Scratchbox? I don't > expect we'd roll anything out that didn't first test OK. I don't think testing > and verifying workness is complicated here. I understand that we can test the rpm, but it seems like the generation and testing overhead is going to be significantly larger than generating a brand new everything rpm and relying on squid.

(In reply to comment #10) > (In reply to comment #9) > > > This > > > will also install files that are going to cross targets which makes it very > > > difficult to build the rpm incrementally. > > > > Why does that make it difficult? Based on earlier comments, you wouldn't be > > working inside of Scratchbox in rpmbuild, so I wouldn't think crossing targets > > would be an issue. > > Generating the list of files to include in the rpm is going to be very > difficult. I don't think it's that hard. You can compare the results of 'find /builds/scratchbox' before and after the install. > > Why don't you feel comfortable that we'll have a working Scratchbox? I don't > > expect we'd roll anything out that didn't first test OK. I don't think testing > > and verifying workness is complicated here. > > I understand that we can test the rpm, but it seems like the generation and > testing overhead is going to be significantly larger than generating a brand > new everything rpm and relying on squid. I don't deny that it's more work, but it has clear benefits: * Reduces network load * Protects the links in the event that a proxy dies when we go to deploy this. * Cleaner implementation

It looks like I can build this in a much cleaner way than I had originally thought. Working from clean installation of scratchbox, the following steps are what I think will get us to a working version of host-python shared between CHINOOK and FREMANTLE su - cd /builds/scratchbox/users/cltbld/targets/FREMANTLE_ARMEL rm -rf usr/local/lib/* usr/local/bin/* usr/local/include/* usr/local/ssl exit /scratchbox/moz_scratchbox wget http://mercurial.selenic.com/release/mercurial-1.5.1.tar.gz wget http://www.python.org/ftp/python/2.5.5/Python-2.5.5.tar.bz2 wget http://www.openssl.org/source/openssl-0.9.8m.tar.gz wget http://www.bzip.org/1.0.5/bzip2-1.0.5.tar.gz mkdir jhford-py cd jhford-py tar zxf ../mercurial-1.5.1.tar.gz tar zxf ../bzip2-1.0.5.tar.gz tar jxf ../Python-2.5.5.tar.bz2 tar jxf ../openssl-0.9.8m.tar.gz sb-conf select CHINOOK-ARMEL-2007 fakeroot apt-get remove python2.5 sed -i -e 's|export PYTHONPATH=/usr/lib/python2.5/site-packages:$PYTHONPATH||' ~/.bashrc rm /usr/bin/hg cd ~/jhford-py cd openssl-0.9.8m CC=host-gcc ./config shared --prefix=/host_usr make -j4 fakeroot make install cd ../bzip2-1.0.5/ make CC=host-gcc PREFIX=/host_usr install make CC=host-gcc -f Makefile-libbz2_so cp libbz2.so.1.0* /host_usr/lib/ CC=host-gcc CXX=host-g++ ./configure --prefix=/host_usr make -j4 make altinstall cd ../mercurial-1.5.1 python2.5 setup.py install --prefix=/host_usr This will be much easier to package as an incremental rpm

(In reply to comment #12) ... > make CC=host-gcc -f Makefile-libbz2_so > cp libbz2.so.1.0* /host_usr/lib/ cd ../Python-2.5.5 > CC=host-gcc CXX=host-g++ ./configure --prefix=/host_usr > make -j4 > make install > cd ../mercurial-1.5.1 > python2.5 setup.py install --prefix=/host_usr Sorry, forgot to be in the correct directory.

Building symbols on a test build using Fremantle worked. Trying hg on CHINOOK-ARMEL-2007 was initially broken. Turns out that there is a random old version of mercurial in the PYTHONPATH that the hg script is trying to use instead of our version 1.5.1. Running find /builds/scratchbox/devkits/ -type d -name 'mercurial' -exec rm -rf {} \; find /builds/scratchbox/devkits/ -type d -name 'hgext' -exec rm -rf {} \; as root fixes the problem.

Ok, I have done a successful fremantle and chinook build using this python. I will write the spec file + puppet manifest change now. [sbox-CHINOOK-ARMEL-2007: ~/mozilla-central/objdir] > make buildsymbols echo building symbol store building symbol store rm -f -rf ./dist/crashreporter-symbols rm -f -f "./dist/fennec-2.0a1pre.en-US.linux-gnueabi-arm.crashreporter-symbols.zip" /home/cltbld/mozilla-central/objdir/config/nsinstall -D ./dist/crashreporter-symbols /host_usr/bin/python2.5 /home/cltbld/mozilla-central/toolkit/crashreporter/tools/symbolstore.py \ -c --vcs-info \ -s /home/cltbld/mozilla-central \ ./dist/host/bin/dump_syms \ ./dist/crashreporter-symbols \ ./dist/bin > \ ./dist/crashreporter-symbols/fennec-2.0a1pre-Linux-20100719130242-symbols.txt Processing file: ./dist/bin/TestDNS Processing file: ./dist/bin/mozilla-xremote-client ./dist/bin/mozilla-xremote-client, section '.debug_frame': the call frame entry at offset 0x158 refers to register 80, whose name we don't know Processing file: ./dist/bin/TestCallbacks <snip> Processing file: ./dist/bin/plugins/libnptest.so echo packing symbols packing symbols /home/cltbld/mozilla-central/objdir/config/nsinstall -D ./dist/ cd ./dist/crashreporter-symbols && \ zip -r9D "../fennec-2.0a1pre.en-US.linux-gnueabi-arm.crashreporter-symbols-full.zip" . <snip> adding: PrimitiveTest/FB101B89188B972C058F7FFE8643C4790/PrimitiveTest.sym (deflated 73%) adding: crashreporter/6A3E755E0CC451CAF2B99DE924CB7CC70/crashreporter.sym (deflated 85%)

This rpm will do the needed cleanup of the scratchbox to proper install a host-arch version of python, its dependencies and things that depend on it.

this is my initial idea. Not sure if it is valid syntax

Comment on attachment 458812 [details] [diff] [review] patch to add a specfile for intalling our host_usr tools for both scratchbox environments that we support Two comments: (From IRC) Don't do all the deletion in %pre since it's a one time thing -- as you suggested, use an exec. This way we can use this RPM on a fresh new system if we need to. Also, I'd prefer that all of .bashrc was synced rather than making an edit like this in the RPM. And a tiny tiny nit: the release is 0moz1. Otherwise, this WFM.

(In reply to comment #18) > Comment on attachment 458812 [details] [diff] [review] > patch to add a specfile for intalling our host_usr tools for both scratchbox > environments that we support > > Two comments: > (From IRC) Don't do all the deletion in %pre since it's a one time thing -- as > you suggested, use an exec. This way we can use this RPM on a fresh new system > if we need to. > > Also, I'd prefer that all of .bashrc was synced rather than making an edit like > this in the RPM. > > And a tiny tiny nit: the release is 0moz1. > > Otherwise, this WFM. Ok, I will do this. I did 0moz2 because i installed the package and wanted to test upgrading.

I have removed some of the cleanup that depends on having had scratchbox-mercurial installed. The cleanup logic included is still valid as long as we continue to use the current scratchbox rpm.

after confirming with Ben on IRC, it looks like I don't even need the following clenup logic from the first patch +rm -rf /builds/scratchbox/users/cltbld/targets/FREMANTLE_ARMEL/usr/local/lib/* +rm -rf /builds/scratchbox/users/cltbld/targets/FREMANTLE_ARMEL/usr/local/bin/* +rm -rf /builds/scratchbox/users/cltbld/targets/FREMANTLE_ARMEL/usr/local/include/* +rm -rf /builds/scratchbox/users/cltbld/targets/FREMANTLE_ARMEL/usr/local/ssl because rpm -e on slaves that have had scratchbox-mercurial will remove those files and slaves that haven't had the package installed will not have ever had those files. I will run the manifest changes attached here through staging when I can and ask for review on them

I have tested this in staging.

Comment on attachment 462830 [details] [diff] [review] updated spec file After IRC discussion, and being told that the files being removed are part of "scratchbox" not "scratchbox-mercurial" I'm generally okay with the removals here. rm is going to return non-zero if they fail though -- will that halt installation? If so, we should guard against that. Also, my comment about .bashrc still stands -- sync the whole thing through Puppet, don't modify it like this.

Comment on attachment 462861 [details] [diff] [review] tested patch Needs the .bashrc still. You probably want require => [Package["scratchbox-mercurial"], Install_rpm["scratchbox"]], otherwise this could fail if we try to do it on a non-ref platform machine.

(In reply to comment #23) > Comment on attachment 462830 [details] [diff] [review] > updated spec file > > After IRC discussion, and being told that the files being removed are part of > "scratchbox" not "scratchbox-mercurial" I'm generally okay with the removals > here. rm is going to return non-zero if they fail though -- will that halt > installation? If so, we should guard against that. I did test that and the package installed after those files were already missing. I could throw in an '|| true' to explicitly ignore the outcome > > Also, my comment about .bashrc still stands -- sync the whole thing through > Puppet, don't modify it like this. Sure, I will do this. (In reply to comment #24) > Comment on attachment 462861 [details] [diff] [review] > tested patch > > Needs the .bashrc still. > You probably want require => [Package["scratchbox-mercurial"], > Install_rpm["scratchbox"]], otherwise this could fail if we try to do it on a > non-ref platform machine. Sounds good, will do that too.

(In reply to comment #25) > (In reply to comment #23) > > Comment on attachment 462830 [details] [diff] [review] [details] > > updated spec file > > > > After IRC discussion, and being told that the files being removed are part of > > "scratchbox" not "scratchbox-mercurial" I'm generally okay with the removals > > here. rm is going to return non-zero if they fail though -- will that halt > > installation? If so, we should guard against that. > > I did test that and the package installed after those files were already > missing. I could throw in an '|| true' to explicitly ignore the outcome The "find"s will pass either way, but the "rm" will return non-zero -- did you test that one, specifically?

rm -f would probably work too :)

This package will set up the scratchbox properly to be able to use a Host Python interpreter for Maemo 4 and replace the installed one in Maemo 5 I am copying the full .bashrc in with this RPM file instead of through puppet because following the RPM deps without our puppet infrastructure will result in an invalid installation otherwise.

I don't know what effect (if any) this will have on the puppet work going in tomorrow.

This RPM does not sync .bashrc as part of the RPM payload. This means that installation without our puppet configs results in a broken HG installation until PYTHONPATH stops being set in the .bashrc file. Puppet patch forthcoming.

haven't tested this yet, but it is using the same statements that install moz_scratchbox. will test on the train today

Comment on attachment 465258 [details] [diff] [review] sync bashrc in puppet This is fine, but you should probably set the 'owner' and 'group attributes, or else it'll be owned by whatever UID owns it on the Puppet master (0:0 or 50something:50something) -- best to be explicit.

Comment on attachment 465255 [details] [diff] [review] rpm that doesn't sync .bashrc http://hg.mozilla.org/build/rpm-sources/rev/d9da1f9917fa

(In reply to comment #32) > Comment on attachment 465258 [details] [diff] [review] > sync bashrc in puppet > > This is fine, but you should probably set the 'owner' and 'group attributes, or > else it'll be owned by whatever UID owns it on the Puppet master (0:0 or > 50something:50something) -- best to be explicit. Ok, I will do that.

Comment on attachment 465258 [details] [diff] [review] sync bashrc in puppet Tested this in staging and works.

Comment on attachment 465258 [details] [diff] [review] sync bashrc in puppet You're going to hate me, but please change the source filename to match the target filename, that's how we've been organizing files on the server. r=me with that change.

(In reply to comment #36) > Comment on attachment 465258 [details] [diff] [review] > sync bashrc in puppet > > You're going to hate me, but please change the source filename to match the > target filename, that's how we've been organizing files on the server. r=me > with that change. sure :) Just didn't want the file to disappear in /N/

same filename for both

I'm landing this for John, gonna grab the bug so I don't forget.

This looks to be landing cleanly.

this is working.