One of the things that I would like us to measure is it turning off PGO has any meaningful impact on the Telemetry measures. Vladan promised to help look into the Telemetry measures to see if they indicate a major impact or not. I'll file another bug on that. Build config overlords: what is the best way to do this? I'd prefer us to come up with a way to do this only for libxul and nothing else, since PGOing other things such as js etc will not have any effect on the question at hand.

I think just putting NO_PROFILE_GUIDED_OPTIMIZE=1 into toolkit/library/Makefile.in will be the simplest thing to do. That will go through all the other steps of a PGO build, including building an optimized omnijar and PGOing the JS DLL, but not libxul.

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #1) > I think just putting NO_PROFILE_GUIDED_OPTIMIZE=1 into > toolkit/library/Makefile.in will be the simplest thing to do. That will go > through all the other steps of a PGO build, including building an optimized > omnijar and PGOing the JS DLL, but not libxul. That won't work, because the object files will be built in the AST form instead of actual object form, and the linker won't be doing LTCG, which it needs to do to transform the AST to machine code.

(In reply to comment #2) > (In reply to Ted Mielczarek [:ted.mielczarek] from comment #1) > > I think just putting NO_PROFILE_GUIDED_OPTIMIZE=1 into > > toolkit/library/Makefile.in will be the simplest thing to do. That will go > > through all the other steps of a PGO build, including building an optimized > > omnijar and PGOing the JS DLL, but not libxul. > > That won't work, because the object files will be built in the AST form instead > of actual object form, and the linker won't be doing LTCG, which it needs to do > to transform the AST to machine code. So do you have a better alternative?

No, the linker is smart enough to figure this out. It will just print out some message like "IL files detected, restarting with /LTCG". The net effect is that you get LTCG, but not PGO.

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #4) > No, the linker is smart enough to figure this out. It will just print out > some message like "IL files detected, restarting with /LTCG". The net effect > is that you get LTCG, but not PGO. But there's not much point not disabling LTCG, since that's what sucks most memory.

Not true, recall bug 710712 comment 7? The abnormally low numbers there were because we broke getting profile data, so we were simply compiling with LTCG.

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #6) > Not true, recall bug 710712 comment 7? The abnormally low numbers there were > because we broke getting profile data, so we were simply compiling with LTCG. Well, that's just a difference of 600-700 megs. If that's all that can be gained, we'll hit the address space limit again after some time with LTCG but without PGO. So, I think it would probably be a good idea to experiment with the more radical approach of turning off LTCG altogether for libxul.

https://hg.mozilla.org/mozilla-central/rev/70baa7e07838 I'll back this out tomorrow.

Thanks a lot for the fast turn-around, Kim!

700MB is nearly 20% of the available 4GB of address space. The reality here is that there's no magic bullet aside from disabling LTCG entirely.

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #11) > 700MB is nearly 20% of the available 4GB of address space. The reality here > is that there's no magic bullet aside from disabling LTCG entirely. OK, I pushed a try patch to actually measure how much LTCG without PGO shaves off. https://tbpl.mozilla.org/?tree=Try&rev=3efc812d488a

Whoa, here's the linker max vmem size on that try push: 107491328 Have I done something wrong?!

Trying something different: https://tbpl.mozilla.org/?tree=Try&rev=de87f513e828

(In reply to Mike Hommey [:glandium] from comment #14) > Trying something different: > https://tbpl.mozilla.org/?tree=Try&rev=de87f513e828 Didn't work, but this should: https://tbpl.mozilla.org/?tree=Try&rev=1b2882bc2412

https://tbpl.mozilla.org/?tree=Try&rev=266f6645a2a5 is already running.

(In reply to Mike Hommey [:glandium] from comment #16) > https://tbpl.mozilla.org/?tree=Try&rev=266f6645a2a5 is already running. linker max vsize: 3052208128 The parent changeset didn't have a pgo build on m-c, but the subsequent changeset did: linker max vsize: 3225350144

Can you clarify exactly what you were attempting to test?

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #18) > Can you clarify exactly what you were attempting to test? LTCG without PGO.

Backed out the patch: https://hg.mozilla.org/mozilla-central/rev/f3d5c6b932ac

OK, lessons learned from these experiments: 1. Disabling PGO and keeping LTCG enabled at the same time reduces the linker memory usage by about 200 megs, so it's not really worth it. 2. Disabling PGO *and* LTCG reduces the linker memory usage to around 1gig, which pretty much means we'll be safe for a long time.