I'd like GCC 4.3 with plugin support to be added to the Linux build slaves. 1) With all the new static analysis passes we're writing, as well as the project branches, I'd like for RelEng to be able to run static-analysis tinderboxes instead of doing it locally. 2) The GCC 4.1 we're currently using has a bug with template instantiation. This will only show up with code using the STL, which Electrolysis does and more of our regular code may end up doing. In addition GCC 4.3 is likely to produce better code than 4.1. I have a refplatform VM and I'm happy to make a tarball with GCC 4.3 with plugin support to be installed in /tools/gcc4.3 After the upgrade we'll need to do some comparative builds to make sure there aren't perf or unit-test regressions before turning this on for Firefox nightly builds.

Note: because this will not be the system compiler, it shouldn't affect any existing builds on any branch (which use /tools/gcc) without an explicit mozconfig change telling it to use the new compiler.

What's the timeline for this? Any special instructions for compiling gcc4.3? We'd want the source tarball and possibly the binaries archived locally somewhere, as long as instructions on how the compilers were built.

* Get http://ftp.gnu.org/gnu/gcc/gcc-4.3.3/gcc-4.3.3.tar.bz2 and unpack it to /tools/gcc-4.3.3/source * Pull the patch series from http://hg.mozilla.org/users/tglek_mozilla.com/gcc-moz-plugin-mq/ * Apply the following patches to the GCC sources using git-apply (patch -p1 will probably work as well): ** gty.diff ** plugin.diff ** parm_attrs.diff ** c_process_decl.diff * mkdir /tools/gcc-4.3.3/build * cd /tools/gcc-4.3.3/build * /tools/gcc-4.3.3/source/configure --prefix=/tools/gcc-4.3.3/installed --enable-__cxa_atexit --enable-languages=c,c++ * make install * The static analysis will require all of /tools/gcc-4.3.3 including the source/build/installed subdirectories I don't need it "now", but I'd like to start using the new compiler for mozilla-central and electrolysis as soon as 1.9.2 branches, early-August.

> * Pull the patch series from > http://hg.mozilla.org/users/tglek_mozilla.com/gcc-moz-plugin-mq/ > * Apply the following patches to the GCC sources using git-apply (patch -p1 > will probably work as well): > ** gty.diff > ** plugin.diff > ** parm_attrs.diff > ** c_process_decl.diff A few questions about these patches... - How often are these patches being updated / how often are we going to need to re-deploy gcc-4.3.3 with a new set of patches? - Do these patches affect the output vs. an unpatched gcc-4.3.3? If we decide to do a future release of Firefox compiled with gcc 4.3.3, are we going to run into problems because of these patches? - Is there work on getting these patches accepted upstream?

These patches are unlikely to need updating... they've been pretty stable for almost a year. It's theoretically possible that the patches may affect output, but practically it's very unlikely... the patches are small and specifcially about enabling static analysis data. All patches have been upstreamed and GCC 4.5 won't need any patchery. They will not be accepted in the 4.3 release stream.

So I've built gcc-4.3.3 with the patches on one of our build machines. The total size of /tools/gcc-4.3.3 is 1.4 GB. We need to trim this down. Here's a bit of a breakdown of sizes (in MB) of some of the larger directories: 94 /tools/gcc-4.3.3/installed 515 /tools/gcc-4.3.3/source 792 /tools/gcc-4.3.3/build 10 /tools/gcc-4.3.3/source/libgcc 14 /tools/gcc-4.3.3/build/prev-i686-pc-linux-gnu 17 /tools/gcc-4.3.3/build/stage1-i686-pc-linux-gnu 21 /tools/gcc-4.3.3/installed/lib 50 /tools/gcc-4.3.3/source/libstdc++-v3 52 /tools/gcc-4.3.3/installed/libexec 139 /tools/gcc-4.3.3/build/i686-pc-linux-gnu 160 /tools/gcc-4.3.3/source/libjava 168 /tools/gcc-4.3.3/build/stage1-gcc 220 /tools/gcc-4.3.3/build/gcc 220 /tools/gcc-4.3.3/build/prev-gcc 256 /tools/gcc-4.3.3/source/gcc

We need to trim that down because it's a per-slave cost? Couldn't it be a shared readonly NFS mount that all the slaves share? Taras, do you know what bits of build we can nuke? Presumably we can nuke build/prev-gcc and build/stage1-gcc and probably all the non-.h files in source, right?

(In reply to comment #6) > So I've built gcc-4.3.3 with the patches on one of our build machines. The > total size of /tools/gcc-4.3.3 is 1.4 GB. We need to trim this down. Here's a > bit of a breakdown of sizes (in MB) of some of the larger directories: > > 94 /tools/gcc-4.3.3/installed need that, but can nuke docs and stuff. I don't think the rest is needed. I don't see dehydra anywhere in there. You'll need the gcc source to rebuild dehydra, but it's probably easier to just rebuild gcc when you need to do that.

No. The static analysis tbox will probably rebuild dehydra every build, and will need GCC on the slave to do so. I really don't think we want to make the GCC build part of the buildbot stuff though. That means we need the GCC build and source directories, right?

(In reply to comment #9) > No. The static analysis tbox will probably rebuild dehydra every build, and > will need GCC on the slave to do so. I really don't think we want to make the > GCC build part of the buildbot stuff though. That means we need the GCC build > and source directories, right? yes. We wont have the disk-space issue with gcc4.5, not that it helps. If someone cares about disk space, we could backport some gcc4.5 patches for this.

I don't really like the idea of 50 or 60 (and growing) linux slaves hitting an NFS partition that hard...if we have to we can either grow the / partition on the slaves to make room, create a separate /tools partition, put gcc 3.3 on /builds...or whatever.

The source tarball I gave you originally is the complete GCC sources. We can reduce the size of that by just installing the C/C++ sources: http://ftp.gnu.org/gnu/gcc/gcc-4.3.3/gcc-core-4.3.3.tar.bz2 http://ftp.gnu.org/gnu/gcc/gcc-4.3.3/gcc-g++-4.3.3.tar.bz2 Also I think when we can trim a lot from the /build directory, specifically: build/prev-gcc build/stage1-gcc

(In reply to comment #12) > Also I think when we can trim a lot from the /build directory, specifically: > > build/prev-gcc > build/stage1-gcc I think dehydra just needs the ,h files from the build directory, everything else can go.

I've got gcc-4.3.3 compiled with the patches above. It's around 600 MB uncompressed now after removing .o files and the prev- and stage1- directories. Can you see if this has all the files you need to build dehydra? http://people.mozilla.org/~catlee/gcc-4.3.3.tar.bz2

I've verified it using the VM... all good!

Any reason why the binaries couldn't be installed to /tools/gcc-4.3.3/bin, and do away with the 'installed' directory? It would keep things consistent with the existing gcc-4.1 installation.

I don't think that's a good idea: you'd have to configure --prefix=/tools/gcc-4.3.3 (instead of /tools/gcc-4.3.3/installed) and you might have conflicts with the build/ and source/ directories in the same root.

I checked, and this needs to be called 'gcc433', you can't have two things called 'gcc' installed.

(In reply to comment #17) > I don't think that's a good idea: you'd have to configure > --prefix=/tools/gcc-4.3.3 (instead of /tools/gcc-4.3.3/installed) and you might > have conflicts with the build/ and source/ directories in the same root. There are no files or directories named 'source' or 'build' in the current 'installed' directory, so it doesn't look like we'd have conflicts.

Not sure what puppet is, but you can change the name using configure arguments: --program-suffix=-4.3.3 To avoid changes in external code which seem more delicate. I still think that mixing up the install prefix with the source and build directories is going to cause more confusion than its worth in consistency.

(In reply to comment #20) > Not sure what puppet is, but you can change the name using configure arguments: > > --program-suffix=-4.3.3 > > To avoid changes in external code which seem more delicate. Puppet is what we're using to deploy these files across our slaves. To have both gcc-4.1 and gcc-4.3.3 installed, we need to give them different package names in the puppet configuration, that's all. We can leave the binary named 'gcc'.

(In reply to comment #3) > I don't need it "now", but I'd like to start using the new compiler for > mozilla-central and electrolysis as soon as 1.9.2 branches, early-August. Do you mean for just static-analysis or do you mean for the actual linux Firefox builds as well?

The Linux Firefox builds.

(In reply to comment #22) > (In reply to comment #3) > > I don't need it "now", but I'd like to start using the new compiler for > > mozilla-central and electrolysis as soon as 1.9.2 branches, early-August. > > Do you mean for just static-analysis or do you mean for the actual linux > Firefox builds as well? (In reply to comment #23) > The Linux Firefox builds. Oh, interesting, we assumed this was for static-analysis builds only. Are there any binary compat issues? Note: FF3.6 (moz-192) is planned to be offered as a *minor* update to FF3.5 users, so we need to know there is no binary compat issues between FF3.5.x and FF3.6.x.

Note, this is not for FF3.6, it's for FF 3.7. GCC 4.3 uses libstdc++.so.6... it's a newer minor version, but we really don't care about binary compat with the Linux builds as long as the majority of our tester community can run them: the distros provide the actual builds for older OSes.

Comment on attachment 391343 [details] [diff] [review] gcc-4.3.3 manifest for puppet I think this is fine, but I want cshields to let us know what the subscribe here does. Corey, does it reinstall the given tool if the subscribed dir changes?

This is ready to go next week, assuming Corey thinks it's ok.

The manifest looks good to me..

Comment on attachment 391343 [details] [diff] [review] gcc-4.3.3 manifest for puppet changeset: 43:f147433c46df

This is getting rolled out via puppet now.

I've verified that this is working (at least on the try slaves)