Closed Bug 939551 Opened 11 years ago Closed 10 years ago

libvpx failed to build with "too many sections" using -GL flag on VS2013

Categories

(Core :: Audio/Video, defect)

x86_64
Windows 8.1
defect
Not set
normal

Tracking

()

RESOLVED DUPLICATE of bug 934984

People

(Reporter: xavier114fch, Unassigned)

Details

Using VS 2013 Ultimate on Windows 8.1 x64. If -GL flag is added to --enable-optimize, libvpx fails to build with "Too many sections". mozmake[5]: Entering directory 'e:/mozilla-inbound/obj/media/libvpx' Too many sections Makefile:433: recipe for target 'asm_enc_offsets.asm' failed mozmake[5]: *** [asm_enc_offsets.asm] Error 1 mozmake[5]: *** Deleting file 'asm_enc_offsets.asm' mozmake[5]: Leaving directory 'e:/mozilla-inbound/obj/media/libvpx' e:/mozilla-inbound/config/recurse.mk:99: recipe for target 'media/libvpx/compile ' failed mozmake[4]: *** [media/libvpx/compile] Error 2 However, with --disable-debug and --disable-debug-symbols in MOZCONFIG, libvpx can be built with -GL enabled, but the resulting build cannot be launched with crash reporter comes up.
I have tried to build with -GL again and it still fails. 2:21.28 mozmake.EXE[5]: Entering directory 'e:/mozilla-inbound/obj/media/libvpx ' 2:21.45 Too many sections 2:21.47 Makefile:113: recipe for target 'vp8_asm_enc_offsets.asm' failed 2:21.47 mozmake.EXE[5]: *** [vp8_asm_enc_offsets.asm] Error 1 2:21.47 mozmake.EXE[5]: *** Deleting file 'vp8_asm_enc_offsets.asm' 2:21.47 mozmake.EXE[5]: Leaving directory 'e:/mozilla-inbound/obj/media/libvpx' 2:21.47 e:/mozilla-inbound/config/recurse.mk:100: recipe for target 'media/libv px/compile' failed 2:21.47 mozmake.EXE[4]: *** [media/libvpx/compile] Error 2 2:21.47 mozmake.EXE[4]: Leaving directory 'e:/mozilla-inbound/obj' 2:21.48 e:/mozilla-inbound/config/recurse.mk:39: recipe for target 'compile' fa iled 2:21.48 mozmake.EXE[3]: *** [compile] Error 2 2:21.48 mozmake.EXE[3]: Leaving directory 'e:/mozilla-inbound/obj' 2:21.48 e:/mozilla-inbound/config/rules.mk:589: recipe for target 'default' fai led 2:21.48 mozmake.EXE[2]: *** [default] Error 2 2:21.49 mozmake.EXE[2]: Leaving directory 'e:/mozilla-inbound/obj' 2:21.49 e:/mozilla-inbound/client.mk:398: recipe for target 'realbuild' failed 2:21.49 mozmake.EXE[1]: *** [realbuild] Error 2 2:21.49 mozmake.EXE[1]: Leaving directory 'e:/mozilla-inbound' 2:21.49 client.mk:185: recipe for target 'build' failed 2:21.49 mozmake.EXE: *** [build] Error 2 2:21.52 271 compiler warnings present. 2
Component: Build Config → Video/Audio
Flags: needinfo?(j)
Version: unspecified → Trunk
would help to see the actual command that fails. could be mozmake problem or HOST_SOURCES (build/make/obj_int_extract.c) fails to compile.
Flags: needinfo?(j)
I have confirmed the problem when building Firefox 28.0 with VS2013 Update 1.
I went to obj/media/libvpx and run mozmake, it gave the following. mozmake[1]: Entering directory 'e:/mozilla-inbound/obj/media/libvpx' ./host_obj_int_extract.exe gas vp8_asm_enc_offsets.obj \ > vp8_asm_enc_offsets.asm Too many sections Makefile:113: recipe for target 'vp8_asm_enc_offsets.asm' failed mozmake[1]: *** [vp8_asm_enc_offsets.asm] Error 1 mozmake[1]: *** Deleting file 'vp8_asm_enc_offsets.asm'
Confirmed, unable to build the new libvpx (and as a result all later Firefox/SM/TB/etc.) on VS2012 also. It seems to me what is done there is: an .obj is built, and then things are extracted from the .obj again with a custom-built tool. That's a little backwards, isn't it? Wouldn't that make things too dependent on what kind of object code a compiler/linker spits out (which may change between compiler versions or even updates of the same VS version)? Output (from Seamonkey comm-release): q:\sm\mozilla\config\rules.mk:982:0$ mozbuild.action.cl main cl -Fohost_obj_int_extract.obj -c -TC -nologo -Fdhost_obj_int_extract.pdb -DXP_WIN32 -DXP_WIN -DWIN32 -D_WIN32 -DNO_X11 -D_CRT_SECURE_NO_W ARNINGS -O2 -fp:precise -arch:SSE2 -GFLTs -GS- -Qfast_transcendentals -Qpar -MD -Iq:/sm/mo zilla/media/libvpx -I. -I../../dist/include -Iq:/sm/build-optimized-suite/mozilla/dist/include/nsp r -Iq:/sm/build-optimized-suite/mozilla/dist/include/nss -Iq:/sm/build-optimized-suite/mozilla/d ist/include/nspr q:/sm/mozilla/media/libvpx/build/make/obj_int_extract.c obj_int_extract.c host_obj_int_extract.exe q:\sm\mozilla\config\rules.mk:778:0$ q:/sm/build-optimized-suite/mozilla/_virtualenv/Scripts/python. exe q:/sm/mozilla/config/expandlibs_exec.py --depend .deps/host_obj_int_extract.exe.pp --target host _obj_int_extract.exe -- link -NOLOGO -OUT:host_obj_int_extract.exe -PDB:host_obj_int_extract.pdb hos t_obj_int_extract.obj -MACHINE:X86 host_obj_int_extract.obj : MSIL .netmodule or module compiled with /GL found; restarting link with / LTCG; add /LTCG to the link command line to improve linker performance Generating code Finished generating code q:\sm\mozilla\config\rules.mk:1506:0$ nsinstall nsinstall -t -m 755 'host_obj_int_extract.exe' '../. ./dist/host/bin' q:\sm\build-optimized-suite\mozilla\media\libvpx\Makefile:150:0$ ./host_obj_int_extract.exe gas vp8_ asm_enc_offsets.obj \ > vp8_asm_enc_offsets.asm Too many sections q:\sm\build-optimized-suite\mozilla\media\libvpx\Makefile:150:0: command './host_obj_int_extract.exe gas vp8_asm_enc_offsets.obj \ > vp8_asm_enc_offsets.asm' failed, return code 1 <quantize_sse4.obj>: Found error <quantize_ssse3.obj>: Found error <media_libvpx.lib.desc>: Found error <media_libvpx.lib.desc>: Found error <libs>: Found error evaluation from q:\sm\mozilla\config\recurse.mk:189:53:7:0: command 'e:/mozilla-build/python/python. exe q:/sm/mozilla/build/pymake/pymake/../make.py -C ../../media/libvpx libs' failed, return code 2 evaluation from q:\sm\mozilla\config\recurse.mk:189:53:12:0: command 'e:/mozilla-build/python/python .exe q:/sm/mozilla/build/pymake/pymake/../make.py ../../db/sqlite3/src_libs ../../media/libjpeg_libs ../../media/libvorbis_libs ../../media/libopus_libs ../../media/libnestegg_libs ../../media/libvpx_ libs ../../media/libogg_libs ../../media/libtheora_libs ../../media/libpng_libs ../../media/kiss_fft _libs ../../media/libcubeb_libs ../../media/libspeex_resampler_libs ../../media/libsoundtouch_libs' failed, return code 2 q:\sm\mozilla\config\recurse.mk:162:0: command 'e:/mozilla-build/python/python.exe q:/sm/mozilla/bui ld/pymake/pymake/../make.py -C config/external libs' failed, return code 2 q:\sm\mozilla\config\rules.mk:602:0: command 'e:/mozilla-build/python/python.exe q:/sm/mozilla/build /pymake/pymake/../make.py libs' failed, return code 2 q:\sm\build-optimized-suite\Makefile:52:0: command 'e:/mozilla-build/python/python.exe q:/sm/mozilla /build/pymake/pymake/../make.py -C mozilla default' failed, return code 2 q:\sm\client.mk:391:0: command 'e:/mozilla-build/python/python.exe q:/sm/mozilla/build/pymake/pymake /../make.py -j4 -C q:/sm/build-optimized-suite' failed, return code 2
(In reply to Mark Straver from comment #5) > It seems to me what is done there is: an .obj is built, and then things are > extracted from the .obj again with a custom-built tool. That's a little > backwards, isn't it? Wouldn't that make things too dependent on what kind of > object code a compiler/linker spits out (which may change between compiler > versions or even updates of the same VS version)? The COFF/PE format has been used by Windows since NT 3.1 (that's before Windows 95, or over two decades ago now). media/libvpx/build/make/obj_int_extract.c:624 contains a check that the file has no more than 96 sections, matching an old limit of the Windows image loader. If you remove that check, does the build succeed?
(In reply to Timothy B. Terriberry (:derf) from comment #6) > The COFF/PE format has been used by Windows since NT 3.1 (that's before > Windows 95, or over two decades ago now). I'm aware of that, but can you reliably trust object code as a source of extracted data? Obviously, as seen in this bug, the number of sections differs depending on compile flags (and compiler version, since the VS version used for official builds doesn't run into this build bustage)... > media/libvpx/build/make/obj_int_extract.c:624 contains a check that the file > has no more than 96 sections, matching an old limit of the Windows image > loader. If you remove that check, does the build succeed? I'll give that a try and report back. Would there be a potential problem when more than 96 sections are present, or is that simply a limit for loading the object file to extract data from that no longer matters?
Removing the check makes the tool crash when actually used: Problem signature: Problem Event Name: APPCRASH Application Name: host_obj_int_extract.exe Application Version: 0.0.0.0 Application Timestamp: 53296056 Fault Module Name: MSVCR110.dll Fault Module Version: 11.0.51106.1 Fault Module Timestamp: 5098858e Exception Code: c0000005 Exception Offset: 000190ff OS Version: 6.1.7601.2.1.0.256.1 Locale ID: 2057 Additional Information 1: 0a9e Additional Information 2: 0a9e372d3b4ad19135b953a78882e789 Additional Information 3: 0a9e Additional Information 4: 0a9e372d3b4ad19135b953a78882e789 I did a quick check and MS documentation (from 2013) still states 96 is the limit. There's no indication this has changed, so 96 is the limit for the image loader. Looks like you may have to change your approach to what this was written for. See http://msdn.microsoft.com/library/windows/hardware/gg463125 output from build window: host_obj_int_extract.exe q:\sm\mozilla\config\rules.mk:778:0$ q:/sm/build-optimized-suite/mozilla/_virtualenv/Scripts/python. exe q:/sm/mozilla/config/expandlibs_exec.py --depend .deps/host_obj_int_extract.exe.pp --target host _obj_int_extract.exe -- link -NOLOGO -OUT:host_obj_int_extract.exe -PDB:host_obj_int_extract.pdb hos t_obj_int_extract.obj -MACHINE:X86 host_obj_int_extract.obj : MSIL .netmodule or module compiled with /GL found; restarting link with / LTCG; add /LTCG to the link command line to improve linker performance Generating code Finished generating code q:\sm\mozilla\config\rules.mk:1506:0$ nsinstall nsinstall -t -m 755 'host_obj_int_extract.exe' '../. ./dist/host/bin' q:\sm\build-optimized-suite\mozilla\media\libvpx\Makefile:152:0$ ./host_obj_int_extract.exe gas vp8_ asm_enc_offsets.obj \ > vp8_asm_enc_offsets.asm *** CRASH *** q:\sm\build-optimized-suite\mozilla\media\libvpx\Makefile:152:0: command './host_obj_int_extract.exe gas vp8_asm_enc_offsets.obj \ > vp8_asm_enc_offsets.asm' failed, return code 5 <quantize_sse4.obj>: Found error <quantize_ssse3.obj>: Found error <media_libvpx.lib.desc>: Found error <media_libvpx.lib.desc>: Found error <libs>: Found error evaluation from q:\sm\mozilla\config\recurse.mk:189:53:7:0: command 'e:/mozilla-build/python/python. exe q:/sm/mozilla/build/pymake/pymake/../make.py -C ../../media/libvpx libs' failed, return code 2 evaluation from q:\sm\mozilla\config\recurse.mk:189:53:12:0: command 'e:/mozilla-build/python/python .exe q:/sm/mozilla/build/pymake/pymake/../make.py ../../db/sqlite3/src_libs ../../media/libjpeg_libs ../../media/libvorbis_libs ../../media/libopus_libs ../../media/libnestegg_libs ../../media/libvpx_ libs ../../media/libogg_libs ../../media/libtheora_libs ../../media/libpng_libs ../../media/kiss_fft _libs ../../media/libcubeb_libs ../../media/libspeex_resampler_libs ../../media/libsoundtouch_libs' failed, return code 2 q:\sm\mozilla\config\recurse.mk:162:0: command 'e:/mozilla-build/python/python.exe q:/sm/mozilla/bui ld/pymake/pymake/../make.py -C config/external libs' failed, return code 2 q:\sm\mozilla\config\rules.mk:602:0: command 'e:/mozilla-build/python/python.exe q:/sm/mozilla/build /pymake/pymake/../make.py libs' failed, return code 2 q:\sm\build-optimized-suite\Makefile:52:0: command 'e:/mozilla-build/python/python.exe q:/sm/mozilla /build/pymake/pymake/../make.py -C mozilla default' failed, return code 2 q:\sm\client.mk:391:0: command 'e:/mozilla-build/python/python.exe q:/sm/mozilla/build/pymake/pymake /../make.py -j4 -C q:/sm/build-optimized-suite' failed, return code 2
I have searched the Dear Web and it sounds like RyanVM has bumped into this on IRC earlier. http://krijnhoetmer.nl/irc-logs/developers/20131105 (search for "host_obj_int_extract.exe") And he has already filed Bug 934984 to turn off optimization for this part as PGO implies -GL.
Flags: needinfo?(ryanvm)
Why am I being needinfo'd here?
Flags: needinfo?(ryanvm)
Status: UNCONFIRMED → RESOLVED
Closed: 10 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.