This is likely a SeaMonkey/Thunderbird bug only and it may go away when merging c-c with m-c. Currently one cannot various mochitest suites, e.g. "mozmake mochitest-browser-chrome" for SeaMonkey because of this error: UnicodeDecodeError: 'ascii' codec can't decode byte 0xad in position 837: ordinal not in range(128) in testing\mozbase\mozprocess\mozprocess\winprocess.py (probably line 154 or near there, I added some debug code). This happens because the test environment (env variables) includes this one from configure.in: 'NONASCII=\xad\xad' (see http://hg.mozilla.org/mozilla-central/annotate/2f198e81ed98//configure.in#l7792) The code in winprocess.py tries to decode the whole byte string to ascii when appending the unicode string u'LD_LIBRARY_PATH=f:\\mozilla\\tree-hg\\comm-central\\obj-i686-pc-mingw32\\mozilla\\dist\\bin' This fails as \xad is not part of ascii standard. Not sure if there is a simple workaround or fix for this problem.

It means mochitest for SeaMonkey will fail on Japanese Windows.

I was unable to build SeaMonkey... 0:32.81 Embedding manifest from h:/m/comm-central/mail/app/thunderbird.exe.mani fest 0:32.82 0:32.82 h:/m/comm-central/mozilla/mail/app/thunderbird.exe.manifest : general e rror c1010070: Failed to load and parse the manifest. ???????????????? 0:32.82 0:32.83 h:/m/comm-central/config/rules.mk:703: recipe for target 'thunderbird.e xe' failed Why Thunderbird is being built when building SeaMonkey??? My .mozconfig is: ac_add_options --enable-application=suite mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../objdir-sm-release ac_add_options --enable-winsdk-directx

That's Bug 1040009

Still fails with patches from bug 1040009: $ ./mozilla/mach build 0:00.28 c:\mozilla-build\mozmake\mozmake.EXE -f client.mk -s 0:01.04 Adding client.mk options from h:/m/comm-central/.mozconfig: 0:01.04 MOZ_OBJDIR=objdir-sm-release 0:01.04 FOUND_MOZCONFIG := h:/m/comm-central/.mozconfig 0:01.94 cd objdir-sm-release 0:01.95 h:/m/comm-central/configure 0:02.16 loading cache ./config.cache 0:02.78 h:/m/comm-central/mozilla/configure: line 84: Ambiguous: command not fo und 0:02.78 Failed eval'ing the following: 0:03.22 Ambiguous object directory detected. We detected that both h:\m\comm-ce ntral\objdir-sm-release and h:\m\comm-central\mozilla\objdir-sm-release could be object directories. This is typically caused by having a mozconfig pointing to a different object directory from the current working directory. To solve this p roblem, ensure you do not have a default mozconfig in searched paths. 0:03.24 *** Fix above errors and then restart with\ 0:03.24 "c:/mozilla-build/mozmake/mozmake.EXE -f client.mk build " 0:03.25 h:/m/comm-central/client.mk:349: recipe for target 'configure' failed 0:03.25 mozmake.EXE[1]: *** [configure] Error 1 0:03.25 client.mk:363: recipe for target 'objdir-sm-release/Makefile' failed 0:03.25 mozmake.EXE: *** [objdir-sm-release/Makefile] Error 2 0:03.25 0 compiler warnings present. 2 I also changed .mozconfig as follows, still no luck: ac_add_options --enable-application=suite mk_add_options MOZ_OBJDIR=objdir-sm-release ac_add_options --enable-winsdk-directx

The build system insists on finding manifest from $SRCDIR/mozilla/suite/app rather than $SRCDIR/suite/app: 0:20.46 Embedding manifest from h:/m/comm-central/suite/app/seamonkey.exe.manif est 0:20.49 0:20.49 h:/m/comm-central/mozilla/suite/app/seamonkey.exe.manifest : general er ror c1010070: Failed to load and parse the manifest. ????????????????

Even tbpl keeps on failing to build SeaMonkey since 22 July. http://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/latest-comm-central-trunk/ http://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/2014-08-12-00-30-04-comm-central-trunk/ I have to conclude that SeaMonkey is virtually dead because no new contributor can build the tree.

BTW: This bug is not for sorting out build problems that are being worked on.

Still fails even with bug 1040009 patches: 3:35.08 creating ./config.status 3:35.65 Traceback (most recent call last): 3:35.65 File "./config.status", line 924, in <module> 3:35.65 config_status(**args) 3:35.65 File "h:\m\comm-central\mozilla\python\mozbuild\mozbuild\config_statu s.py", line 134, in config_status 3:35.65 emitter = TreeMetadataEmitter(env) 3:35.65 File "h:\m\comm-central\mozilla\python\mozbuild\mozbuild\frontend\emi tter.py", line 84, in __init__ 3:35.65 mozinfo.find_and_update_from_json(config.topobjdir) 3:35.65 File "h:\m\comm-central\mozilla\testing\mozbase\mozinfo\mozinfo\mozin fo.py", line 153, in find_and_update_from_json 3:35.65 build = MozbuildObject.from_environment() 3:35.66 File "h:\m\comm-central\mozilla\python\mozbuild\mozbuild\base.py", li ne 181, in from_environment 3:35.66 raise ObjdirMismatchException(topobjdir, _config_topobjdir) 3:35.66 mozbuild.base.ObjdirMismatchException: Objdir mismatch: h:\m\comm-centr al\objdir-sm-release != h:\m\comm-central\mozilla\objdir-sm-release 3:35.67 *** Fix above errors and then restart with\ 3:35.67 "c:/mozilla-build/mozmake/mozmake.EXE -f client.mk build " 3:35.68 h:/m/comm-central/client.mk:361: recipe for target 'configure' failed 3:35.68 mozmake.EXE[1]: *** [configure] Error 1 3:35.68 client.mk:375: recipe for target 'h:/m/comm-central/objdir-sm-release/M akefile' failed 3:35.69 mozmake.EXE: *** [h:/m/comm-central/objdir-sm-release/Makefile] Error 2 3:35.70 0 compiler warnings present. 2

Hi Masatoshi, this bug is about a UnicodeDecodeError in mozprocess. If you believe you have build issues that are not covered by bug 1040009, please file a new bug under 'Core/Build Config'. Frank, I'd guess that the UnicodeDecodeError is happening here: http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/mozprocess/mozprocess/winprocess.py#142 Changing that line to: values = ["%s=%s" % (key.decode('utf8', 'replace'), value.decode('utf8', 'replace')) Will probably fix your issue. That being said, that line might not be the best place to do the decoding. Ideally we'd decode wherever those strings are first input into the program. Could you let me know if that change works and if so I'll look into making a proper patch? Thanks for finding this!

I was trying to build SeaMonkey because I would have to build the tree to run "mozmake mochitest-browser-chrome". If someone else will work on this bug, I will not investigate further. (In reply to Andrew Halberstadt [:ahal] from comment #9) > Changing that line to: > values = ["%s=%s" % (key.decode('utf8', 'replace'), value.decode('utf8', > 'replace')) > > Will probably fix your issue. I'm not sure it will decode the environment string correctly on Windows. Windows ANSI codepage is not UTF-8.

I was finally able to build the tree with MOZ_OBJDIR completely removed! This patch will decode strings using the 'mbcs' encoding which means the ANSI codepage on Windows. This file is Windows-specific anyway as the name implies.

Thanks, that patch looks workable, but I'd prefer to do the decoding a little earlier on to avoid this type of problem again in the future. Does this patch work for you?

(In reply to Andrew Halberstadt [:ahal] from comment #12) > Created attachment 8482879 [details] [diff] [review] > mozprocess_windows_decode > > Thanks, that patch looks workable, but I'd prefer to do the decoding a > little earlier on to avoid this type of problem again in the future. Does > this patch work for you? It worked for me, but I'd rather use sys.getfilesystemencoding() rather than the hard-coded encoding names. It will give us a consistent result for all platforms.

Thanks for the patch and sorry for the incredible late feedback (real life :), attachment 8482879 [details] [diff] [review] works fine and fixes the problem.

What needs to be done to get this checked in to the tree?

I haven't tried, but the patch is likely bit-rotted by now. Then it needs to be reviewed, and I'd also like to see a passing try run with the syntax: try: -a

Please attach a new patch with comments addressed.

According to Google sys.getfilesystemencoding() can return None. On *nix it can return anything e.g. "ANSI_X3.4-1968"

Is this the right approach? I've never used Python before. (In reply to Andrew Halberstadt [:ahal] from comment #16) > I haven't tried, but the patch is likely bit-rotted by now. Then it needs to > be reviewed, and I'd also like to see a passing try run with the syntax: Who would be a suitable reviewer? > try: -a The try server doesn't do anything when I do try: -a This try server job uses: "try: -b do -p all -u all -t none" https://treeherder.mozilla.org/#/jobs?repo=try&revision=10c84ba38901

Comment on attachment 8587430 [details] [diff] [review] v1.patch using sys.getfilesystemencoding() Review of attachment 8587430 [details] [diff] [review]: ----------------------------------------------------------------- Close, but if sys.getfilesystemencoding() returns None, then that will fail. I'd change it to: sys.getfilesystemencoding() or 'utf-8'.

try-server results: https://treeherder.mozilla.org/#/jobs?repo=try&revision=86029fe3ce2a > Close, but if sys.getfilesystemencoding() returns None, then that will fail. > I'd change it to: > sys.getfilesystemencoding() or 'utf-8'. Done.

Comment on attachment 8588070 [details] [diff] [review] v2.patch using sys.getfilesystemencoding() or utf-8 Review of attachment 8588070 [details] [diff] [review]: ----------------------------------------------------------------- Encoding changes always scare me, but I guess this looks good assuming your try run was all green.

> Encoding changes always scare me, but I guess this looks good assuming your > try run was all green. Thanks. Managed to get a all green try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=b7e8d67b2e37 http://hg.mozilla.org/integration/mozilla-inbound/rev/acb01dce9ffe

Comment on attachment 8588070 [details] [diff] [review] v2.patch using sys.getfilesystemencoding() or utf-8 Yeah, it works! Thanks to everyone being involved here.