Closed
Bug 1367995
Opened 7 years ago
Closed 6 years ago
Building geckodriver on local Windows builds fails with: failed to run custom build command for `miniz-sys v0.1.9`
Categories
(Remote Protocol :: Marionette, defect, P3)
Tracking
(firefox-esr52 unaffected, firefox53 unaffected, firefox54 unaffected, firefox55 fixed)
RESOLVED
WORKSFORME
Tracking | Status | |
---|---|---|
firefox-esr52 | --- | unaffected |
firefox53 | --- | unaffected |
firefox54 | --- | unaffected |
firefox55 | --- | fixed |
People
(Reporter: Paenglab, Assigned: ato)
References
Details
Attachments
(2 files)
(deleted),
text/plain
|
Details | |
(deleted),
patch
|
Details | Diff | Splinter Review |
0:56.06 error: failed to run custom build command for `miniz-sys v0.1.9`
0:56.09 process didn't exit successfully: `z:\Mozilla\TB\testing\geckodriver\.\release\build\miniz-sys-0d42c5494ba5bc47\build-script-build` (exit code: 101)
0:56.10 --- stdout
0:56.10 TARGET = Some("i686-pc-windows-msvc")
0:56.10 OPT_LEVEL = Some("3")
0:56.10 TARGET = Some("i686-pc-windows-msvc")
0:56.10 HOST = Some("x86_64-pc-windows-msvc")
0:56.10 TARGET = Some("i686-pc-windows-msvc")
0:56.10 TARGET = Some("i686-pc-windows-msvc")
0:56.10 HOST = Some("x86_64-pc-windows-msvc")
0:56.11 CC_i686-pc-windows-msvc = None
0:56.11 CC_i686_pc_windows_msvc = None
0:56.11 TARGET_CC = None
0:56.11 CC = None
0:56.12 TARGET = Some("i686-pc-windows-msvc")
0:56.12 HOST = Some("x86_64-pc-windows-msvc")
0:56.12 CFLAGS_i686-pc-windows-msvc = None
0:56.12 CFLAGS_i686_pc_windows_msvc = None
0:56.12 TARGET_CFLAGS = None
0:56.12 CFLAGS = None
0:56.13 PROFILE = Some("release")
0:56.13 running: "cl.exe" "/nologo" "/MD" "/O2" "/Foz:\\Mozilla\\TB\\testing\\geckodriver\\.\\i686-pc-windows-msvc\\release\\build\\miniz-sys-18005000ddedadf4\\out\\miniz.o" "/c" "miniz.c"
0:56.13
0:56.13
0:56.13 failed to execute command: Das System kann die angegebene Datei nicht finden. (os error 2)
0:56.13 Is `cl.exe` not installed? (see https://github.com/alexcrichton/gcc-rs#compile-time-requirements for help)
I tried to set the path explained in https://github.com/alexcrichton/gcc-rs#compile-time-requirements, but didn't success.
What do I need to do? I'm not a compiler specialist.
Comment 1•7 years ago
|
||
Me too. Unable to compile suite. Is probably caused by Bug 1340637. From the patch it looks like
it doesn't take the comm-central dir into accound and tries to find the source in the topmost sourcedir:
/Foz:\\Mozilla\\TB\\testing\\
Should probably be /Foz:\\Mozilla\\TB\\mozilla\\testing\\ in case of comm-central builds. Best locally would probably to have an option to disable it. I see not much if any use for it locally. That still would still ned a fix for the offcial TB and SM builds if it is needed there.
Comment 2•7 years ago
|
||
Andreas and Ted, it seems that changes in bug 1340637 busted local builds, at least for Thunderbird for some people.
I tried local Firefox and Thunderbird 32bit debug builds and they seem to work.
Richard, can you please attach a longer log of the first 56 seconds to see where the error happens. My build is happily compiling ;-)
Flags: needinfo?(ted)
Flags: needinfo?(ato)
Reporter | ||
Comment 3•7 years ago
|
||
I don't have the log anymore. But before where no errors. If needed I have a older log (the initial build which failed) with output before the error.
Comment 4•7 years ago
|
||
I spoke too early, it fails for me, too, after eight(!!) minutes.
If it's a "top source dir" issue, then FF local build won't be affected. I'm just wondering why server builds on treeherder still work for C-C. I've just retriggered a Mac opt
https://treeherder.mozilla.org/#/jobs?repo=comm-central&revision=abc0bcd3caec68343fce1ff4fcb700edd6c72076&selectedJob=102237328
and that worked.
Comment 5•7 years ago
|
||
>> /Foz:\\Mozilla\\TB\\testing\\geckodriver
This is the object dir so I take it back that it might be a toplevel source problem. If the script tries to run the x86 cl.exe for cross compile it might miss some files from the x64 version. The server uses the tooltool package and might not be affected. Anyway just speculation.
Assignee | ||
Comment 6•7 years ago
|
||
A quick search indicates cl.exe is not exposed in the Windows command prompt by default unless you enable the ‘SDK command line’ or install the C++ requirements in Visual Studio. Possibly you need to start a Visual Studio command prompt that sets all the correct path variables for it to pick up cl.exe correctly.
We need cl.exe to compile some C libraries as part of a Rust dependency used by testing/geckodriver.
Note that with https://bugzilla.mozilla.org/show_bug.cgi?id=1368035 we are making the compilation of testing/geckodriver opt-in rather than mandatory on local developer machines. Compiling it in CI will still be required as it is used to run the WPT wdspec tests.
Flags: needinfo?(ato)
Comment 7•7 years ago
|
||
I local Firefox build fails as well. So basically you've broken all local compiles on Windows for FF, TB and SM.
So can you please come up with a fix or workaround real quick.
Severity: normal → blocker
Flags: needinfo?(ato)
Summary: error: failed to run custom build command for `miniz-sys v0.1.9` → local Windows builds for Firefox and Thunderbird broken: error: failed to run custom build command for `miniz-sys v0.1.9`
Updated•7 years ago
|
Hardware: Unspecified → All
Assignee | ||
Comment 8•7 years ago
|
||
I am temporarily disabling compilation of testing/geckodriver in https://bugzilla.mozilla.org/show_bug.cgi?id=1368084 until https://bugzilla.mozilla.org/show_bug.cgi?id=1368035, to make it opt-in for local developers, is fixed.
Assignee: nobody → ato
Status: NEW → ASSIGNED
Depends on: 1368084
Flags: needinfo?(ted)
Flags: needinfo?(ato)
Updated•7 years ago
|
Blocks: 1340637
Keywords: regression
Comment 10•7 years ago
|
||
(In reply to Andreas Tolfsen ‹:ato› from comment #6)
> A quick search indicates cl.exe is not exposed in the Windows command prompt
> by default unless you enable the ‘SDK command line’ or install the C++
> requirements in Visual Studio. Possibly you need to start a Visual Studio
> command prompt that sets all the correct path variables for it to pick up
> cl.exe correctly.
The problem is not this. As I mentioned in bug 1367935 comment 4, I'm able to build geckodriver via cargo directly from the source code in the same environment, which means that it is mach which hides the necessary environment variables.
It reminds me of bug 1350001 which removed almost all environment variables for Windows when calling into cargo, which then broke stylo build (see bug 1355464). They could be related.
Comment 11•7 years ago
|
||
Comment 12•7 years ago
|
||
(In reply to Jorg K (GMT+2) from comment #11)
> Created attachment 8871793 [details] [diff] [review]
> Patch by FRG to get the compile going again ;-) - final solution may differ.
That's the same workaround I came up with in bug 1367935 comment 2 :)
Assignee | ||
Comment 13•7 years ago
|
||
(In reply to Xidorn Quan [:xidorn] UTC+10 from comment #10)
> (In reply to Andreas Tolfsen ‹:ato› from comment #6)
> > A quick search indicates cl.exe is not exposed in the Windows command prompt
> > by default unless you enable the ‘SDK command line’ or install the C++
> > requirements in Visual Studio. Possibly you need to start a Visual Studio
> > command prompt that sets all the correct path variables for it to pick up
> > cl.exe correctly.
>
> The problem is not this. As I mentioned in bug 1367935 comment 4, I'm able
> to build geckodriver via cargo directly from the source code in the same
> environment, which means that it is mach which hides the necessary
> environment variables.
>
> It reminds me of bug 1350001 which removed almost all environment variables
> for Windows when calling into cargo, which then broke stylo build (see bug
> 1355464). They could be related.
This is very interesting information. I will follow up on this.
In the meantime I’m resolving this bug since the build problems on m-c are fixed by https://bugzilla.mozilla.org/show_bug.cgi?id=1368084.
Assignee | ||
Updated•7 years ago
|
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Comment 14•7 years ago
|
||
> This is very interesting information. I will follow up on this.
If you need to call cl.exe directly also take into account the the x86 cross compiler needs the x64 libs and that VS2017 also has a different directory structure.
https://bugzilla.mozilla.org/show_bug.cgi?id=1318143#c34
Comment 15•7 years ago
|
||
xidorn (or someone else who can reproduce this): can you `cargo run` this test app from the same shell where you're building Firefox? It's just using the same version of the gcc crate that we have vendored, and printing the MSVC paths it returns.
On my Windows machine, which has MSVC 2015 and 2017 installed, in a MozillaBuild start-shell.bat shell it prints:
CC_x86_64-pc-windows-msvc = None
CC_x86_64_pc_windows_msvc = None
HOST_CC = None
CC = None
CFLAGS_x86_64-pc-windows-msvc = None
CFLAGS_x86_64_pc_windows_msvc = None
HOST_CFLAGS = None
CFLAGS = None
64-bit MSVC path: "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64\\cl.exe"
CC_i686-pc-windows-msvc = None
CC_i686_pc_windows_msvc = None
TARGET_CC = None
CC = None
CFLAGS_i686-pc-windows-msvc = None
CFLAGS_i686_pc_windows_msvc = None
TARGET_CFLAGS = None
CFLAGS = None
32-bit MSVC path: "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64_x86\\cl.exe"
Flags: needinfo?(xidorn+moz)
Comment 16•7 years ago
|
||
Oops, the test app is here: https://github.com/luser/rust-msvc-test
Comment 17•7 years ago
|
||
$ cargo run
Updating registry `https://github.com/rust-lang/crates.io-index`
Downloading gcc v0.3.42
Compiling gcc v0.3.42
Compiling msvc-test v0.1.0 (file:///J:/temp/rust-msvc-test-master/rust-msvc-test-master)
Finished dev [unoptimized + debuginfo] target(s) in 12.78 secs
Running `target\debug\msvc-test.exe`
CC_x86_64-pc-windows-msvc = None
CC_x86_64_pc_windows_msvc = None
HOST_CC = None
CC = None
CFLAGS_x86_64-pc-windows-msvc = None
CFLAGS_x86_64_pc_windows_msvc = None
HOST_CFLAGS = None
CFLAGS = None
64-bit MSVC path: "d:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64\\cl.exe"
CC_i686-pc-windows-msvc = None
CC_i686_pc_windows_msvc = None
TARGET_CC = None
CC = None
CFLAGS_i686-pc-windows-msvc = None
CFLAGS_i686_pc_windows_msvc = None
TARGET_CFLAGS = None
CFLAGS = None
32-bit MSVC path: "d:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64\\cl.exe"
(I installed MSVC 2015 on D: because my C: is a SSD)
Comment 18•7 years ago
|
||
This seems wrong:
32-bit MSVC path: "d:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64\\cl.exe"
You don't have any MSVC environment variables set in that shell, do you?
Comment 19•7 years ago
|
||
What are "MSVC environment variables"? I'm just running start-shell-msvc2015-x64.bat.
Comment 20•7 years ago
|
||
If I run start-shell-msvc2015.bat, I got:
$ cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.4 secs
Running `target\debug\msvc-test.exe`
CC_x86_64-pc-windows-msvc = None
CC_x86_64_pc_windows_msvc = None
HOST_CC = None
CC = None
CFLAGS_x86_64-pc-windows-msvc = None
CFLAGS_x86_64_pc_windows_msvc = None
HOST_CFLAGS = None
CFLAGS = None
64-bit MSVC path: "d:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64_x86\\cl.exe"
CC_i686-pc-windows-msvc = None
CC_i686_pc_windows_msvc = None
TARGET_CC = None
CC = None
CFLAGS_i686-pc-windows-msvc = None
CFLAGS_i686_pc_windows_msvc = None
TARGET_CFLAGS = None
CFLAGS = None
32-bit MSVC path: "d:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64_x86\\cl.exe"
Comment 21•7 years ago
|
||
By the way, I backed out bug 1368084 and I got the following error:
1:58.69 failed to execute command: 隰厄ソス・ョ螢ケ・・ケァ蠕娯螺郢晁シ斐<郢ァ・、郢晢スォ邵コ迹夲スヲ荵昶命邵コ荵晢ス顔クコ・セ邵コ蟶呻ス鍋クイ・ス(os error 2)
Looks like a wrong encoding is used.
Updated•7 years ago
|
status-firefox53:
--- → unaffected
status-firefox54:
--- → unaffected
status-firefox55:
--- → fixed
status-firefox-esr52:
--- → unaffected
Target Milestone: --- → mozilla55
Comment 22•7 years ago
|
||
start-shell-msvc2015-x64.bat gives me:
$ cargo run
Compiling gcc v0.3.42
Compiling msvc-test v0.1.0 (file:///C:/mozilla-source/rust-msvc-test)
Finished dev [unoptimized + debuginfo] target(s) in 3.21 secs
Running `target\debug\msvc-test.exe`
CC_x86_64-pc-windows-msvc = None
CC_x86_64_pc_windows_msvc = None
HOST_CC = None
CC = None
CFLAGS_x86_64-pc-windows-msvc = None
CFLAGS_x86_64_pc_windows_msvc = None
HOST_CFLAGS = None
CFLAGS = None
64-bit MSVC path: "c:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64\\cl.exe"
CC_i686-pc-windows-msvc = None
CC_i686_pc_windows_msvc = None
TARGET_CC = None
CC = None
CFLAGS_i686-pc-windows-msvc = None
CFLAGS_i686_pc_windows_msvc = None
TARGET_CFLAGS = None
CFLAGS = None
32-bit MSVC path: "c:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64\\cl.exe"
Seems to be similar to emk's.
Flags: needinfo?(xidorn+moz)
Comment 23•7 years ago
|
||
(In reply to Andreas Tolfsen ‹:ato› from comment #13)
> In the meantime I’m resolving this bug since the build problems on m-c are
> fixed by https://bugzilla.mozilla.org/show_bug.cgi?id=1368084.
Is there a bug that tracks the underlying issue so that we can re-enable geckodriver building on Windows?
I don't think it's a good idea to continue works on a closed bug.
Comment 24•7 years ago
|
||
Comment 25•7 years ago
|
||
Apparently it does not take care of failures in local builds.
Assignee | ||
Comment 26•7 years ago
|
||
(In reply to Masatoshi Kimura [:emk] from comment #23)
> (In reply to Andreas Tolfsen ‹:ato› from comment #13)
> > In the meantime I’m resolving this bug since the build problems on m-c are
> > fixed by https://bugzilla.mozilla.org/show_bug.cgi?id=1368084.
>
> Is there a bug that tracks the underlying issue so that we can re-enable
> geckodriver building on Windows?
We can use this bug to track that, certainly.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee | ||
Updated•7 years ago
|
Severity: blocker → normal
Component: Build Config → Marionette
Keywords: regression
Product: Core → Testing
Summary: local Windows builds for Firefox and Thunderbird broken: error: failed to run custom build command for `miniz-sys v0.1.9` → Building geckodriver on local Windows builds fails with: failed to run custom build command for `miniz-sys v0.1.9`
Version: unspecified → Trunk
Assignee | ||
Updated•7 years ago
|
Target Milestone: mozilla55 → ---
Comment 27•7 years ago
|
||
Oh, I got the correct result if I run start-shell.bat:
$ cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.4 secs
Running `target\debug\msvc-test.exe`
CC_x86_64-pc-windows-msvc = None
CC_x86_64_pc_windows_msvc = None
HOST_CC = None
CC = None
CFLAGS_x86_64-pc-windows-msvc = None
CFLAGS_x86_64_pc_windows_msvc = None
HOST_CFLAGS = None
CFLAGS = None
64-bit MSVC path: "D:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64\\cl.exe"
CC_i686-pc-windows-msvc = None
CC_i686_pc_windows_msvc = None
TARGET_CC = None
CC = None
CFLAGS_i686-pc-windows-msvc = None
CFLAGS_i686_pc_windows_msvc = None
TARGET_CFLAGS = None
CFLAGS = None
32-bit MSVC path: "D:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64_x86\\cl.exe"
But the current latest release of MozillaBuild does not support building from start-shell.bat yet. I have to use start-shell-msvc2015(-x64).bat.
Comment 28•7 years ago
|
||
(In reply to Masatoshi Kimura [:emk] from comment #27)
> But the current latest release of MozillaBuild does not support building
> from start-shell.bat yet. I have to use start-shell-msvc2015(-x64).bat.
Hm, I was wrong. I could build Firefox with geckodriver enabled.
Comment 29•7 years ago
|
||
start-shell-msvc2015(-x64).bat will set $VCINSTALLDIR, so the tool detection will not run:
https://github.com/alexcrichton/gcc-rs/blob/0d87a9b739a9b0086015d5356de729f1dc1fb010/src/windows_registry.rs#L70-L74
Updated•7 years ago
|
Priority: -- → P3
Assignee | ||
Comment 30•6 years ago
|
||
I think this problem has been resolved. At least we have had no
complaints that enabling geckodriver in the default local build has
caused any problems for Windows developers.
Status: REOPENED → RESOLVED
Closed: 7 years ago → 6 years ago
Resolution: --- → WORKSFORME
Updated•2 years ago
|
Product: Testing → Remote Protocol
You need to log in
before you can comment on or make changes to this bug.
Description
•