I'm seeing this today trying to locally run: # ./mach mochitest-remote dom/camera/test/ This worked yesterday -- all that's changed is I updated my trees and rebuilt the emulator. FWIW, u'\u279c'[1] is a character in my command prompt, but it always has been. 1. http://www.fileformat.info/info/unicode/char/279C/index.htm

The problem goes away if I change my PROMPT envvar: export PROMPT=#

Please paste the traceback :). This problem can usually be fixed by finding the offending string and explicitly encoding it: my_str.encode('utf-8', 'replace')

I'll get you an update as soon as the dependent bug is fixed. :)

As requested.

(I guess the question is: why is the script slurping my entire environment? This is likely to cause random problems for lots of people, no?)

Mozrunner is probably setting env=os.environ somewhere. This can be useful if people want to set environment that affects the subprocess (i.e gecko). Did this used to work for you? If so I imagine this is a regression from bug 1050561 (ugh, I hate taking unicode patches).

Yep, confirmed that bug 1050561 caused the regression.

This patch fixes the problem, but it might re-expose the problem in bug 1050561. Frank, could you test this out to make sure it doesn't unfix bug 1050561 please?

Comment on attachment 8592929 [details] [diff] [review] Fix UnicodeEncodeError With a bit of delay I got bad news: This seems to expose the other bug again. Output of "mozmake mochitest-browser-chrome" after the skipped tests: dir: addon-sdk/test Traceback (most recent call last): File "_tests/testing/mochitest/runtests.py", line 2623, in <module> main() File "_tests/testing/mochitest/runtests.py", line 2617, in main return_code = mochitest.runTests(options) File "_tests/testing/mochitest/runtests.py", line 2120, in runTests result = self.runMochitests(options, tests_in_dir, onLaunch) File "_tests/testing/mochitest/runtests.py", line 2040, in runMochitests result = self.doTests(options, onLaunch, testsToRun) File "_tests/testing/mochitest/runtests.py", line 2175, in doTests self.manifest = self.buildProfile(options) File "_tests/testing/mochitest/runtests.py", line 1432, in buildProfile certificateStatus = self.fillCertificateDB(options) File "_tests/testing/mochitest/runtests.py", line 1301, in fillCertificateDB [certutil, "-N", "-d", certdbPath, "-f", pwfilePath], env=toolsEnv) File "_tests/testing/mochitest/runtests.py", line 273, in call process.run() File "f:\mozilla\tree-hg\comm-central\mozilla\testing\mozbase\mozprocess\mozprocess\processhandler.py", line 728, in run self.proc = self.Process([self.cmd] + self.args, **args) File "f:\mozilla\tree-hg\comm-central\mozilla\testing\mozbase\mozprocess\mozprocess\processhandler.py", line 115, in __init__ universal_newlines, startupinfo, creationflags) File "c:\mozilla-build\python\Lib\subprocess.py", line 711, in __init__ errread, errwrite) File "f:\mozilla\tree-hg\comm-central\mozilla\testing\mozbase\mozprocess\mozprocess\processhandler.py", line 268, in _execute_child winprocess.EnvironmentBlock(env), File "f:\mozilla\tree-hg\comm-central\mozilla\testing\mozbase\mozprocess\mozprocess\winprocess.py", line 145, in _ _init__ self._as_parameter_ = LPCWSTR("\0".join(values)) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 9: ordinal not in range(128) f:/mozilla/tree-hg/comm-central/mozilla/../suite/build.mk:30: recipe for target 'mochitest-browser-chrome' failed

Sorry, this slipped through the cracks. Thanks for tweeting about it mhoye! The root of the problem is that some things are expecting unicode strings, and others are expecting byte strings. I think the best way forward here is to move the fix from bug 1050561 out of processhandler.py and into winprocess.py. I'll get a new patch up.

I believe this patch fixes both this and bug 1050561. Here's a try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=c1f7162ea6af

Review commit: https://reviewboard.mozilla.org/r/41023/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/41023/

Comment on attachment 8732153 [details] MozReview Request: Bug 1152428 - [mozprocess] Fix UnicodeEncodeError when non-ascii characters are in the environment, r?wlach https://reviewboard.mozilla.org/r/41023/#review37637 This looks fine! Thanks.