https://tbpl.mozilla.org/php/getParsedLog.php?id=46429721&tree=Try jitprofiling.obj : error LNK2001: unresolved external symbol _PgoRTClientUnload It's not just jitprofiling.obj (if I disable PGO, the error just moves to the next file) and it's not just mozjs.dll (if I disable PGO, the error just moves to the next binary). My local builds are fine. This must be something in the automation config.

Try emptying build/mozconfig.cache.

(If that works, that just means that it's one more reason to have specific PGO jobs on try)

Disregard comment 3, that was a mozconfig error (see bug 1009807 comment 63)

These PGO errors look a lot like the ones we used to see before Update 2. Turns out, some builders don't have Update 2! Even the successful builds are not quite on the official release. For reference: 18.00.30723 is Update 3 18.00.30501 is Update 2 18.00.30220 is ??? Maybe the Update 2 CTP or RC? 18.00.21005.1 is ??? Something even older Successful PGO builds (search "18.00" in the build logs): https://tbpl.mozilla.org/php/getParsedLog.php?id=47005546&tree=Try&full=1 18.00.30220 for x86 https://tbpl.mozilla.org/php/getParsedLog.php?id=47007220&tree=Try&full=1 18.00.30220 for x64 Failed PGO builds: https://tbpl.mozilla.org/php/getParsedLog.php?id=47003586&tree=Try&full=1 18.00.21005.1 for x86 https://tbpl.mozilla.org/php/getParsedLog.php?id=47011110&tree=Try&full=1 18.00.21005.1 for x86 https://tbpl.mozilla.org/php/getParsedLog.php?id=47001904&tree=Try&full=1 18.00.21005.1 for x64

Can you help figure out what's up with those machines? If you load VS2013 and go to Help/About, it should tell you what update VS thinks it's on.

> Failed PGO builds: > https://tbpl.mozilla.org/php/getParsedLog.php?id=47003586&tree=Try&full=1 > 18.00.21005.1 for x86 I am on buildduty this week so I am not as responsive. FAILED PGO BUILD slave: b-2008-ix-0026. seems to be: Version 12.0.21005.1 REL log: https://tbpl.mozilla.org/php/getParsedLog.php?id=47003586&tree=Try&full=1 (from above) screenshot: http://people.mozilla.org/~jlund/Screen%20Shot%202014-08-29%20at%206.06.28%20PM.png from about/help: Microsoft Visual Studio Premium 2013 Version 12.0.21005.1 REL Microsoft .NET Framework Version 4.5.50938 SUCCESSFUL PGO BUILD log: https://tbpl.mozilla.org/php/getParsedLog.php?id=47005546&tree=Try&full=1 18.00.30220 for x86 slave: b-2006-ix-0035 screenshot: http://people.mozilla.org/~jlund/Screen%20Shot%202014-08-29%20at%206.34.34%20PM.png from about/help: Microsoft Visual Studio Premium 2013 Version 12.0.30219.00 Update 2 CTP Microsoft .NET Framework Version 4.5.50938 Looks like we have Update 2 working but this REL fails. Does this help? Q, do you know how it's possible we have ended up with two different vs2013 versions?

Are there any machines at all with 30501? CTP is Community Technology Preview, so basically 30219 is a beta. For production use we'll need to have the released version of Update 2. (Although, if it turns out that we need to re-deploy anyway, we might as well get Update 3.)

Thanks to bug 1068922, PGO is now completely green: https://tbpl.mozilla.org/?tree=Try&rev=7db869f03470 Microsoft (R) C/C++ Optimizing Compiler Version 18.00.30723 for x86 Microsoft (R) C/C++ Optimizing Compiler Version 18.00.30723 for x64