r157958 fixed PR12979 which was casing problems with our CheckedInt.h. I will upload the patches in a sec.

The new patch is because the version of glibc on centos 5 does not define O_CLOEXEC.

Testing on 32 bit linux found llvm.org/pr13048. I have fixed an restarted the tests.

32 bit linux is OK now, so I have checked in the patches: https://hg.mozilla.org/build/rpm-sources/rev/d9b3401ac0ba https://hg.mozilla.org/build/braindump/rev/662fbcab17f8 will attach the puppet ones in a sec.

This is not needed to build with xcode 4.3, but is with xcode 4.0. If I remember correctly, we have xcode 4.1 in the bots. I don't know if it needs this patch or not. Would you mind testing and r+ or r- the patch depending on the result? Thanks

Comment on attachment 631522 [details] [diff] [review] support for building with "old" xcode Yeah, this needed even on the lion builders...

Comment on attachment 631522 [details] [diff] [review] support for building with "old" xcode https://hg.mozilla.org/build/braindump/rev/049fb30ef0b3

I'm still waiting for some outstanding release related jobs. I hope to start deploying later today or tomorrow.

Comment on attachment 631371 [details] [diff] [review] puppet change 1: install the new packages Sorry about the delay, we had a couple of chemspill releases last week. I'll land the patch and deploy the packages today. BTW, would you consider to use tooltool (https://github.com/jhford/tooltool) to deploy clang packages next time? This will require building relocatable tarballs with clang (not rpms and dmgs), in a similar way how we build gonk gor b2g builds.

Comment on attachment 631371 [details] [diff] [review] puppet change 1: install the new packages http://hg.mozilla.org/build/puppet-manifests/rev/587ce9e7e906

> BTW, would you consider to use tooltool (https://github.com/jhford/tooltool) > to deploy clang packages next time? This will require building relocatable > tarballs with clang (not rpms and dmgs), in a similar way how we build gonk > gor b2g builds. That lets us use the same format for OS X and linux? That is awesome! I will try it next time, yes!

This should only be checked in after a try run confirms the new packages are OK.

(In reply to Rafael Ávila de Espíndola (:espindola) from comment #14) > That lets us use the same format for OS X and linux? That is awesome! I will > try it next time, yes! yes, you can use multiple versions and use tooltool manifests in the source tree to point to the particular version. Let's use it next time!

Comment on attachment 634152 [details] [diff] [review] puppet change 2: remove the old packages lgtm, let me know when you want me to deploy this.

A try push is at https://tbpl.mozilla.org/?tree=Try&rev=c2228487f25a It looks like there is a problem in mochitest-4. It is not clear if it is a regression in clang or if new firefox code found the problem. I did a push with the old version in https://tbpl.mozilla.org/?tree=Try&rev=ca9e8fc2105f to find out.

The failure on try in caused by llvm's r158126, which was fixed by r158160, so we just missed it :-( Rail, what do you think we should do? If the old packages were still in place I would say just close this bug as "wontfix" and I would open a new one with that bug (and another one Ehsan found) fixed. Since the OS X packages have been replaced, I can see these options: * Rollback the OS X packages. * Live with this bug for now. Might cause some confusion, but this is still not in production, so we can probably live with it. * Rebuild the packages at r158160. Somewhat annoying since we know we will do another upgrade and we want to change the package format.

What do you think about this: 1) rail backs out http://hg.mozilla.org/build/puppet-manifests/rev/587ce9e7e906 2) espindola bui1ds 58160 for linux, linux64 and mac do we can use it by tooltool. We'll need to package the files into a tarball and unpack it by setup.sh (so it will be accessible from /builds/slave/xxxx/clang(-version?). Try is aware of tooltool manifests (there are empty ones in browser/config/tooltool-manifests), but we'll need to ask IT to put the package and setup.sh to http://runtime-binaries.pvt.build.mozilla.org/tooltool (someday we'll have a webapp for this! :) ). What do you think?

OK, so you suggestion is to upgrade to 158160 using tooltool. I can do that. Is there documentation somewhere on what exactly the tarball has to look like?

There is a Wiki page here: https://wiki.mozilla.org/ReleaseEngineering/Applications/Tooltool The main idea is having a simple tool which fetches files and runs a command to setup the environment properly. From my understanding you'll need to build a relocatable package and have a setup.sh script to unpack it and probably setup symlinks (ln -s clang-xxx clang), then update your manifests and mozconfigs. You may want to take a look at the b2g build logs to see how they deploy gonk toolchain.

The build is still going, but we should be able to do the code review in parallel. This script is a reduced version of the generic toolchain building one. In is *not* reproducible, but the spec file was not. This version is linux (centos 5) only. I will update it to support os X as soon as the linux build finishes.

* The old tar in centos has no --mtime * Install in "clang", not "inst" * We are only doing a two stage build. With these changes, it is possible to build a clang.tar.bz2 in centos 5.

Not tested yet.

FTR. pkgdmg provider is a little bit dumb, and doesn't cleanup in /var/db after upgrading a package. It was thinking that r155417 is installed because there is /var/db/.puppet_pkgdmg_installed_clang-3.0-r155417.moz0.dmg left after r155417 -> r158158 upgrade. I removed /var/db/.puppet_pkgdmg_installed_clang-3.0-r155417.moz0.dmg on bld-lion-r5-0{01..40} (try lion build slaves). They should pickup r155417 today.

Comment on attachment 635293 [details] [diff] [review] rm 158158 Worked fine in staging.

necessary changes: * Different set of patches * Different host compiler and options * The tar in OS X is in /usr/bin and doesn't support --owner

Comment on attachment 635293 [details] [diff] [review] rm 158158 http://hg.mozilla.org/build/puppet-manifests/rev/40f20eb317ca deployed

Comment on attachment 635313 [details] [diff] [review] Add support to OS X in the build script lgtm

Bug 766708 is still ongoing, so I uploaded the clang packages to http://people.mozilla.org/~respindola. Rail, can you deploy them to relengweb1.dmz.scl3.mozilla.com or do you have to build them again?

FTR, $ wget https://raw.github.com/jhford/tooltool/master/tooltool.py $ wget https://hg.mozilla.org/integration/mozilla-inbound/raw-file/84dffec9742c/build/unix/build-clang/setup.sh $ wget http://people.mozilla.org/~respindola/clang-linux.tar.bz2 $ wget http://people.mozilla.org/~respindola/clang-osx.tar.bz2 $ python2.6 tooltool.py -m linux.manifest add clang-linux.tar.bz2 setup.sh $ python2.6 tooltool.py -m osx.manifest add clang-osx.tar.bz2 setup.sh $ sed -i 's/clang-.*.tar.bz2/clang.tar.bz2/g' *.manifest $ cat linux.manifest [ { "size": 47, "digest": "2005a41fe97a5e00997063705f39d42b6a43b1cf7ba306cbc7b1513de34cdcd050fc6326efa2107f19ba0cc67914745dbf13154fa748010a93cf072481ef4aaa", "algorithm": "sha512", "filename": "setup.sh" }, { "size": 71534797, "digest": "66bd11bea6e1f07090e9e03c833e107088097605611fd455e80b280ce2b71ca71ff9841a66614f62da162469b222b5eefd5535373b199c60fd485959889b5dcb", "algorithm": "sha512", "filename": "clang.tar.bz2" } ] $ cat osx.manifest [ { "size": 47, "digest": "2005a41fe97a5e00997063705f39d42b6a43b1cf7ba306cbc7b1513de34cdcd050fc6326efa2107f19ba0cc67914745dbf13154fa748010a93cf072481ef4aaa", "algorithm": "sha512", "filename": "setup.sh" }, { "size": 63015959, "digest": "e0ac132a77b052d6494d4bf02f17aba475138ffcff07a5f92f17f45b1d5f27b7b0cea36c29473965271e60910af82ffd4989df94c10d24794d1bf8362bcb785b", "algorithm": "sha512", "filename": "clang.tar.bz2" } ] $ sudo cp -vi setup.sh /var/www/html/runtime-binaries/tooltool/sha512/2005a41fe97a5e00997063705f39d42b6a43b1cf7ba306cbc7b1513de34cdcd050fc6326efa2107f19ba0cc67914745dbf13154fa748010a93cf072481ef4aaa `setup.sh' -> `/var/www/html/runtime-binaries/tooltool/sha512/2005a41fe97a5e00997063705f39d42b6a43b1cf7ba306cbc7b1513de34cdcd050fc6326efa2107f19ba0cc67914745dbf13154fa748010a93cf072481ef4aaa' $ sudo cp -vi clang-linux.tar.bz2 /var/www/html/runtime-binaries/tooltool/sha512/66bd11bea6e1f07090e9e03c833e107088097605611fd455e80b280ce2b71ca71ff9841a66614f62da162469b222b5eefd5535373b199c60fd485959889b5dcb `clang-linux.tar.bz2' -> `/var/www/html/runtime-binaries/tooltool/sha512/66bd11bea6e1f07090e9e03c833e107088097605611fd455e80b280ce2b71ca71ff9841a66614f62da162469b222b5eefd5535373b199c60fd485959889b5dcb' $ sudo cp -vi clang-osx.tar.bz2 /var/www/html/runtime-binaries/tooltool/sha512/e0ac132a77b052d6494d4bf02f17aba475138ffcff07a5f92f17f45b1d5f27b7b0cea36c29473965271e60910af82ffd4989df94c10d24794d1bf8362bcb785b `clang-osx.tar.bz2' -> `/var/www/html/runtime-binaries/tooltool/sha512/e0ac132a77b052d6494d4bf02f17aba475138ffcff07a5f92f17f45b1d5f27b7b0cea36c29473965271e60910af82ffd4989df94c10d24794d1bf8362bcb785b'

I forgot to build a 32 bit linux package. I will fix that in a sec, but I think we are also missing a chmod: retry: Calling <function run_with_timeout at 0x2aaaaed12ed8> with args: (['/tools/tooltool.py', '--url', 'http://runtime-binaries.pvt.build.mozilla.org/tooltool', '--overwrite', '-m', 'browser/config/tooltool-manifests/linux64/releng.manifest', 'fetch'], 1260, None, None, False, True), kwargs: {}, attempt #1 Executing: ['/tools/tooltool.py', '--url', 'http://runtime-binaries.pvt.build.mozilla.org/tooltool', '--overwrite', '-m', 'browser/config/tooltool-manifests/linux64/releng.manifest', 'fetch'] ERROR - failed to fetch 'setup.sh': HTTP Error 403: Forbidden ERROR - failed to fetch 'clang.tar.bz2': HTTP Error 403: Forbidden ERROR - The following files failed: 'setup.sh', clang.tar.bz2'

The linux32 package is at http://people.mozilla.org/~respindola/clang-linux32.tar.bz2

$ wget http://people.mozilla.org/~respindola/clang-linux32.tar.bz2 $ python2.6 tooltool.py -m linux32.manifest add clang-linux32.tar.bz2 setup.sh $ sed -i 's/clang-.*.tar.bz2/clang.tar.bz2/g' linux32.manifest $ cat linux32.manifest [ { "size": 47, "digest": "2005a41fe97a5e00997063705f39d42b6a43b1cf7ba306cbc7b1513de34cdcd050fc6326efa2107f19ba0cc67914745dbf13154fa748010a93cf072481ef4aaa", "algorithm": "sha512", "filename": "setup.sh" }, { "size": 72350187, "digest": "7d2fbe08aca3ae740e33b8aee872705a3b5229681dd0617ceffd6619fba75cb3cb7e1c3a071218f7cfd464003e5cd773cd8e67d16f78df9c50218fb6671580c6", "algorithm": "sha512", "filename": "clang.tar.bz2" } ] $ sudo cp -vi clang-linux32.tar.bz2 /var/www/html/runtime-binaries/tooltool/sha512/7d2fbe08aca3ae740e33b8aee872705a3b5229681dd0617ceffd6619fba75cb3cb7e1c3a071218f7cfd464003e5cd773cd8e67d16f78df9c50218fb6671580c6 `clang-linux32.tar.bz2' -> `/var/www/html/runtime-binaries/tooltool/sha512/7d2fbe08aca3ae740e33b8aee872705a3b5229681dd0617ceffd6619fba75cb3cb7e1c3a071218f7cfd464003e5cd773cd8e67d16f78df9c50218fb6671580c6' $ sudo chmod 644 /var/www/html/runtime-binaries/tooltool/sha512/*

https://hg.mozilla.org/mozilla-central/rev/c42521d54b79 https://hg.mozilla.org/mozilla-central/rev/84dffec9742c

A new try push with all tests and talos is at https://tbpl.mozilla.org/?tree=Try&rev=fca5d53846e3

OK, looks like it works :-) Should we remove the old clang packages now or should we fix 767508 first?

(In reply to Rafael Ávila de Espíndola (:espindola) from comment #41) > OK, looks like it works :-) \o/ > Should we remove the old clang packages now or should we fix 767508 first? Yeah, let's remove it. I hope bug 767508 will be very simple to solve with $topsrcdir.

TODO: 1) test in staging 2) deploy when Rafael is OK with package removal

Comment on attachment 635868 [details] [diff] [review] remove clang packages OK. Just wait for me to send an email on how to use the new packages.

Add the current clang manifests to the tree. This makes it easy to build a patch that uses them on try, just copy them over the releng.manifest.

Comment on attachment 635868 [details] [diff] [review] remove clang packages http://hg.mozilla.org/build/puppet-manifests/rev/fd06119e5fa6

Feel free to close the bug.

Fixed :-)