Closed Bug 1457583 Opened 7 years ago Closed 2 years ago

Build fails: error: failed to acquire jobserver token: Bad file descriptor (os error 9)

Categories

(Firefox Build System :: General, defect)

3 Branch
x86_64
Linux
defect

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: jimb, Unassigned)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

I'm building Mozilla Central rev b62ad926cf2a on Fedora 26 on x86_64, and the build died with the following message: 0:00.64 Clobber not needed. 0:00.64 Adding make options from /home/jimb/moz/ebg/.mozconfig MOZ_OBJDIR=/home/jimb/moz/ebg/obj-bug OBJDIR=/home/jimb/moz/ebg/obj-bug FOUND_MOZCONFIG=/home/jimb/moz/ebg/.mozconfig export FOUND_MOZCONFIG 0:00.66 /usr/bin/gmake -f client.mk -s 0:01.94 Elapsed: 0.01s; From dist/private: Kept 0 existing; Added/updated 0; Removed 0 files and 0 directories. 0:01.94 Elapsed: 0.01s; From dist/public: Kept 0 existing; Added/updated 0; Removed 0 files and 0 directories. 0:01.97 Elapsed: 0.01s; From dist/xpi-stage: Kept 9 existing; Added/updated 0; Removed 0 files and 0 directories. 0:02.06 Elapsed: 0.19s; From dist/idl: Kept 873 existing; Added/updated 0; Removed 0 files and 0 directories. 0:02.55 Elapsed: 0.56s; From _tests: Kept 994 existing; Added/updated 0; Removed 0 files and 0 directories. 0:02.93 Elapsed: 1.00s; From dist/bin: Kept 2081 existing; Added/updated 0; Removed 0 files and 0 directories. 0:03.43 Elapsed: 1.44s; From dist/include: Kept 5360 existing; Added/updated 0; Removed 0 files and 0 directories. 0:04.33 Elapsed: 0.14s; From ../../dist/idl: Kept 873 existing; Added/updated 0; Removed 0 files and 0 directories. 0:05.15 js/src 0:35.65 force-cargo-library-build 0:35.65 security/sandbox/linux/reporter 0:35.65 security/sandbox/linux/broker 0:35.65 security/sandbox/common 0:35.68 security/sandbox/linux/gtest 0:37.98 Compiling unicode-xid v0.1.0 0:39.47 thread '<unnamed>' panicked at 'failed to acquire jobserver token: Bad file descriptor (os error 9)', librustc_trans/back/write.rs:1835:29 0:39.47 stack backtrace: 0:39.47 error: failed to acquire jobserver token: Bad file descriptor (os error 9) 0:39.48 0: 0x7fb31d35a76f - std::sys::unix::backtrace::tracing::imp::unwind_backtrace::h2fb330a06456ac7e 0:39.48 at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49 0:39.48 1: 0x7fb31d3270f0 - std::sys_common::backtrace::print::hf6ccec7cfd12ed67 0:39.48 error: aborting due to previous error 0:39.48 at libstd/sys_common/backtrace.rs:71 0:39.48 at libstd/sys_common/backtrace.rs:59 0:39.48 2: 0x7fb31d32c573 - std::panicking::default_hook::{{closure}}::h464de9af91e4ca8e 0:39.48 at libstd/panicking.rs:205 0:39.48 3: 0x7fb31d32c2fd - std::panicking::default_hook::hcde1a6167c711c5f 0:39.48 at libstd/panicking.rs:221 0:39.48 4: 0x7fb319757145 - rustc::util::common::panic_hook::h4530c027f4a88a1b 0:39.48 5: 0x7fb31d32cc0c - std::panicking::rust_panic_with_hook::h8dcdd9a7e80a2917 0:39.48 at libstd/panicking.rs:461 0:39.48 6: 0x7fb31d32c810 - std::panicking::begin_panic_fmt::haaf6e293fe070518 0:39.48 at libstd/panicking.rs:344 0:39.49 error: Could not compile `unicode-xid`. 0:39.49 To learn more, run the command again with --verbose. 0:39.50 gmake[4]: *** [/home/jimb/moz/ebg/config/rules.mk:949: force-cargo-library-build] Error 101 0:39.51 gmake[3]: *** [/home/jimb/moz/ebg/config/recurse.mk:73: toolkit/library/rust/target] Error 2 0:39.51 gmake[3]: *** Waiting for unfinished jobs.... 1:23.59 gmake[2]: *** [/home/jimb/moz/ebg/config/recurse.mk:33: compile] Error 2 1:23.59 gmake[1]: *** [/home/jimb/moz/ebg/config/rules.mk:418: default] Error 2 1:23.59 gmake: *** [client.mk:172: build] Error 2 1:23.60 0 compiler warnings present. 1:23.62 ccache (direct) hit rate: 0.0%; (preprocessed) hit rate: 0.0%; miss rate: 100.0%
It seems to be intermittent; I tried 'mach build' again, and it seemed to work okay. This blog post by Paul Smith (GNU Make maintainer) describes some circumstances under which make closes the file descriptors for subprocesses. Perhaps rustc's jobserver implementation isn't prepared for those circumstances? http://make.mad-scientist.net/papers/jobserver-implementation/
Hm what version of rustc are you using? Pieces around this have been fixed relatively recently, although there may be some remaining!
Flags: needinfo?(jimb)
I'm not able to reproduce this, but I'll re-open if it comes up again.
Status: NEW → RESOLVED
Closed: 7 years ago
Flags: needinfo?(jimb)
Resolution: --- → WORKSFORME
I'm now seeing this whenever I try to run configure. The failing command works fine when I run it locally. DEBUG: rustc: Trying rustc INFO: /usr/bin/rustc INFO: checking for cargo... DEBUG: cargo: Trying cargo INFO: /usr/bin/cargo INFO: checking rustc version... DEBUG: Executing: `/usr/bin/rustc --version --verbose` INFO: 1.29.1 INFO: checking cargo version... DEBUG: Executing: `/usr/bin/cargo --version --verbose` INFO: 1.29.0 DEBUG: Executing: `/usr/bin/rustc --print target-list` DEBUG: Creating `/tmp/conftestRkjYSg.rs` with content: DEBUG: | pub extern fn hello() { println!("Hello world"); } DEBUG: Executing: `/usr/bin/rustc --crate-type staticlib --target=x86_64-unknown-linux-gnu -o /tmp/conftest7Vx1zH.rlib /tmp/conftestRkjYSg.rs` DEBUG: The command returned non-zero exit status 1. DEBUG: Its error output was: DEBUG: | thread '<unnamed>' panicked at 'failed to acquire jobserver token: Bad file descriptor (os error 9)', librustc_codegen_llvm/back/write.rs:1824:29 DEBUG: | note: Run with `RUST_BACKTRACE=1` for a backtrace. DEBUG: | error: failed to acquire jobserver token: Bad file descriptor (os error 9) DEBUG: | DEBUG: | error: aborting due to previous error DEBUG: | ERROR: Cannot compile for x86_64-pc-linux-gnu with /usr/bin/rustc Seeing as this prevents me from doing my job, marking critical.
Severity: normal → critical
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
Note: This started after I upgraded to rust 1.29.1. Downgrading to 1.29.0 didn't help. I can't downgrade to 1.28.* because the build system won't let me.
It happening on configure is different from comment 0. I've had this happen in a very specific case: with a wrapper Makefile invoking configure, and with that Makefile being invoked with -j<n>.
Apparently this has something to do with configure running rustc with fd 4 opened to $OBJDIR/config.log. Hacking util.configure to run it with a wrapper that closes that fd makes the problem go away.
Running rustc with an open fd 4 sometimes causes it to fail with an obscure error when it tries to connect to a job server on that fd. Closing the fd solves the problem. close_fds does what we need, and is the default value in Python 3, but not in Python 2.
Blocks: 1497976
Version: Version 3 → 3 Branch

In the process of migrating remaining bugs to the new severity system, the severity for this bug cannot be automatically determined. Please retriage this bug using the new severity system.

Severity: critical → --

If I recall correctly, there was some bug in the jobserver crate back then, which has been fixed since.

Status: REOPENED → RESOLVED
Closed: 7 years ago2 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: