Running a clobber "mach build" of m-c on a Linux box inside a screen session gives me an error that looks like the attached dump. The last line (which indicates the position and ordinal) may vary from one run to the next. I modified mach_commands.py and stuck in a |print(repr(msg))| just above the self.fh.write call that borks, so as to get the actual msg that triggers the error. In this example, the message was: u'\x1b[34m 1:13.57\x1b[m\x0f /home/kats/zspace/mozilla-git-2/media/mtransport/third_party/nrappkit/src/log/r_log.c: In function \ufffd\ufffd\ufffdr_log_register\ufffd\ufffd\ufffd:' Presumably the compiler is injecting those \ufffd [1] replacement characters into its warning message (or whatever it's about to spit out) and mach/python doesn't deal with those nicely? [1] http://www.fileformat.info/info/unicode/char/0fffd/index.htm

Boo. I thought we fixed this a long, long time ago. You sure this is on a recent changeset?

Yeah this was recent m-c.

I think I'm hitting similar issue at the moment rendering me not to be able to make a build for Fennec. Issuing `mach build` results in [hg:globe_with_mag-1079182]☿ mach build 0:00.34 /usr/bin/gmake -f client.mk -s Error running mach: ['build'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: UnicodeEncodeError: 'ascii' codec can't encode character u'\u263f' in position 10: ordinal not in range(128) File "~/mozilla-central/python/mozbuild/mozbuild/mach_commands.py", line 387, in build silent=not verbose) File "~/mozilla-central/python/mozbuild/mozbuild/base.py", line 552, in _run_make return fn(**params) File "~/mozilla-central/python/mozbuild/mozbuild/base.py", line 595, in _run_command_in_srcdir return self.run_process(cwd=self.topsrcdir, **args) File "~/mozilla-central/python/mach/mach/mixin/process.py", line 136, in run_process p.run() File "~/mozilla-central/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 727, in run self.proc = self.Process([self.cmd] + self.args, **args) File "~/mozilla-central/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 114, in __init__ universal_newlines, startupinfo, creationflags) File "/usr/lib64/python2.7/subprocess.py", line 710, in __init__ errread, errwrite) File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child raise child_exception

Greg, do you know what's going on here?

To add more info, I'm on Fedora 22 beta. This started happening after recent update to my system. Issuing wget -q https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py && python bootstrap.py I selected option 2 to build Firefox for Android it ended up with this error. Package autoconf213-2.13-34.fc22.noarch is already installed, skipping. Package mercurial-3.3.3-1.fc22.x86_64 is already installed, skipping. Dependencies resolved. Nothing to do. Complete! Traceback (most recent call last): File "bootstrap.py", line 163, in <module> sys.exit(main(sys.argv)) File "bootstrap.py", line 154, in main dasboot.bootstrap() File "/tmp/tmpVaien9/mozboot/bootstrap.py", line 134, in bootstrap 'content. Like --repo, you should not need to set this.') File "/tmp/tmpVaien9/mozboot/base.py", line 125, in install_mobile_android_packages NotImplementedError: Cannot bootstrap Firefox for Android: mozboot.base does not yet implement install_mobile_android_packages()

Comment #3 and comment #5 are separate bugs. Comment #5 should be filed as a separate bug and assigned to nalexander. Just to confirm, what revision of mozilla-central is encountering this error? `hg log -r .`.

(In reply to Gregory Szorc [:gps] from comment #6) > Comment #3 and comment #5 are separate bugs. Comment #5 should be filed as a > separate bug and assigned to nalexander. Moved to a new Bug 1158443 > > Just to confirm, what revision of mozilla-central is encountering this > error? `hg log -r .`. Status of `hg log -r .` is changeset: 241031:d94e535e171c │ tag: fx-team │ user: Jordan Santell <jsantell@gmail.com> │ date: Mon Apr 13 18:00:02 2015 -0700 │ summary: Bug 1147945 - Let the profiler's buffer size and sample rate be configurab│ le via prefs. r=vp Not sure. Even fresh clone of central gives the error.

Same issue here - seem that ohmyzsh themes that use any unicode symbols or emojis will cause this problem. mach build 0:00.36 /usr/bin/make -f client.mk MOZ_PARALLEL_BUILD=7 -s Error running mach: ['build'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: UnicodeEncodeError: 'ascii' codec can't encode character u'\u25cf' in position 42: ordinal not in range(128) File "/Users/marcos/gecko/python/mozbuild/mozbuild/mach_commands.py", line 387, in build silent=not verbose) File "/Users/marcos/gecko/python/mozbuild/mozbuild/base.py", line 552, in _run_make return fn(**params) File "/Users/marcos/gecko/python/mozbuild/mozbuild/base.py", line 595, in _run_command_in_srcdir return self.run_process(cwd=self.topsrcdir, **args) File "/Users/marcos/gecko/python/mach/mach/mixin/process.py", line 136, in run_process p.run() File "/Users/marcos/gecko/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 727, in run self.proc = self.Process([self.cmd] + self.args, **args) File "/Users/marcos/gecko/testing/mozbase/mozprocess/mozprocess/processhandler.py", line 114, in __init__ universal_newlines, startupinfo, creationflags) File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__ errread, errwrite) File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child raise child_exception ^CError in atexit._run_exitfuncs: Traceback (most recent call last): File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/atexit.py", line 24, in _run_exitfuncs Process Process-1: Traceback (most recent call last): File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap func(*targs, **kargs) File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/util.py", line 325, in _exit_function p.join() File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 145, in join self.run() File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "/Users/marcos/gecko/testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py", line 56, in _collect res = self._popen.wait(timeout) File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/forking.py", line 154, in wait while not pipe.poll(sleep_interval): return self.poll(0) File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/forking.py", line 135, in poll pid, sts = os.waitpid(self.pid, flag) KeyboardInterrupt KeyboardInterrupt Error in sys.exitfunc: Traceback (most recent call last): File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/util.py", line 325, in _exit_function p.join() File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 145, in join res = self._popen.wait(timeout) File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/forking.py", line 154, in wait return self.poll(0) File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/forking.py", line 135, in poll pid, sts = os.waitpid(self.pid, flag) KeyboardInterrupt

STR are needed here. How exactly is Unicode being introduced to the build environment via ohmyzsh? Through an environment variable? If so, which one?

(In reply to Gregory Szorc [:gps] from comment #9) > STR are needed here. > > How exactly is Unicode being introduced to the build environment via > ohmyzsh? Through an environment variable? If so, which one? Not sure, tbh :( I can't see much wrong with line 387 of mach_commands.py... but then again, I know next to nothing about mach or how that code works.

I'm on zshell as well with custom .zshrc file. I'm not using ohmyzsh but I'm manipulating the right prompt which ohmyzsh does also. Commenting out this line [1] in my .zshrc file passes this error. I'm not exactly sure why though. [1]: https://github.com/eyedol/tools/blob/master/zshrc#L112

My guess is your PS1 variable contains non-ascii. But... so does mine. Somehow a Python unicode type is getting passed down into process execution land. subprocess.py is attempting to coerce that unicode into a str using the default conversion encoding of "ascii" and failing. So, the question is: where is the unicode type coming from. Try this: 1) Open python/mozbuild/mozbuild/base.py 2) In _run_make() immediately before "return fn(**params)", insert a new line: "import pdb; pdb.set_trace()" 3) Run mach and hit the breakpoint 4) Type "pp params" 5) Find string values that have the |u'| prefix on them. That denotes a Unicode type and will likely point to our culprit.

So, I think you are right... this is what I get for the following: echo $PS1: %{%f%b%k%}$(build_left_prompt) And then... note the "", which I can't copy/paste because iTerm is using a custom font (souce code powerline)... but there are at least 4 of them: echo $(build_left_prompt) %{%K{black}%}%{%F{011}%} %(!.%{%F{yellow}%}.)marcos@%m%{%K{blue}%F{black}%}%{%F{black}%} %~%{%K{yellow}%F{blue}%}%{%F{black}%} %F{black}%f %F{black}%F{black}splashscreen%f %F{black}●%f %F{black}⍟ 9%f%f %{%k%F{yellow}%}%{%f%}

Added screenshot of theme "powerlevel9k". Repo: https://github.com/bhilburn/powerlevel9k

Yup!. This is what I have with the breakpoint. -> return fn(**params) (Pdb) pp params {u'append_env': {'MACH': u'1'}, u'args': [u'/usr/bin/gmake', u'-f', u'client.mk', u'-s'], u'ensure_exit_code': False, u'explicit_env': None, u'ignore_children': True, u'line_handler': <bound method BuildOutputManager.on_line of <mach.commands.05100728ed2911e4b3248c705a82b138.BuildOutputManager object at 0x7fcc5e6980d0>>, u'log_level': 20, u'pass_thru': False, u'require_unix_environment': False}

Mine I supposed is this ☿; My echo $PS1 [%{%}hg%{%}:!^%{%}]☿

I can confirm this on my Arch Linux box.

So IIUC, before calling a subprocess we convert all the environment key/value from "bytes" data to system / unicode strings. This is possibly submitting non-ASCII strings. This patch removes the decoding and seems to workaround the problem for me. However, it's possible that environment values will then not be passed correctly to subprocess, so I'm not sure this patch is quite correct. BTW in my case, the culprit was CHANGE_LOG_NAME = "Frédéric Wang".

(In reply to Frédéric Wang (:fredw) from comment #18) > BTW in my case, the culprit was CHANGE_LOG_NAME = "Frédéric Wang". (...used for WebKit contribution https://www.webkit.org/coding/contributing.html)

(In reply to Frédéric Wang (:fredw) from comment #18) I had this problem on openSUSE and the workaround patch helped. thanks!

Just had a new contributor report this. STR is to set their PS1 prompt to the Euro symbol instead of the dollar sign.

The comments from 11 months ago are a regression from bug 1050561, and fixing it is tracked by bug 1152428. Unfortunately the fix breaks comm-central builds apparently. I won't dupe this bug, as I'm unclear whether the initial problem filed 2 years ago is still relevant or not.