User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:106.0) Gecko/20100101 Firefox/106.0

Steps to reproduce:

1. Prepare macOS build environment: https://firefox-source-docs.mozilla.org/setup/macos_build.html
2. But use Python 3.11.0 via pyenv (git version)
3. ./mach build

Actual results:

Build fails with pyenv CPython 3.11.0

0:01.27 SyntaxError: Can't build lexer
0:01.28 ./application.ini.h.stub
0:01.29 make[3]: *** [xpcom/idl-parser/xpidl/.deps/xpidl.stub.stub] Error 1
0:01.29 make[3]: *** Waiting for unfinished jobs....
0:01.41 make[2]: *** [export] Error 2
0:01.42 make[1]: *** [default] Error 2
0:01.42 make: *** [build] Error 2
0:01.44 0 compiler warnings present.
0:01.47 Failed to parse ccache stats output: 'Cacheable calls: 933 / 933 (100.0%)/Hits' '0 / 933 ( 0.00%)'

Expected results:

Build should have continued, same as pyenv CPython 3.10.8

The Bugbug bot thinks this bug should belong to the 'Firefox Build System::General' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

(In reply to Kyungjoon Lee from comment #0)

Created attachment 9300763 [details]
python_3.11.0_firefox_build.log

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:106.0) Gecko/20100101 Firefox/106.0

Steps to reproduce:

1. Prepare macOS build environment: https://firefox-source-docs.mozilla.org/setup/macos_build.html
2. But use Python 3.11.0 via pyenv (git version)
3. ./mach build

Actual results:

Build fails with pyenv CPython 3.11.0

0:01.27 SyntaxError: Can't build lexer
0:01.28 ./application.ini.h.stub
0:01.29 make[3]: *** [xpcom/idl-parser/xpidl/.deps/xpidl.stub.stub] Error 1
0:01.29 make[3]: *** Waiting for unfinished jobs....
0:01.41 make[2]: *** [export] Error 2
0:01.42 make[1]: *** [default] Error 2
0:01.42 make: *** [build] Error 2
0:01.44 0 compiler warnings present.
0:01.47 Failed to parse ccache stats output: 'Cacheable calls: 933 / 933 (100.0%)/Hits' '0 / 933 ( 0.00%)'

Expected results:

Build should have continued, same as pyenv CPython 3.10.8

It'll be a while before I will have a chance to look at this and fix it. There's still problems with parts of Mach on Python3.10 as well. For now I'd recommend using Python3.9.

This looks relevant:

 0:47.58 ERROR: c:\users\sasch\documents\github\gecko-dev\xpcom\idl-parser\xpidl\xpidl.py:1641: Invalid regular expression for rule 't_multilinecomment'. global flags not at the start of the expression at position 26
 0:47.58 ERROR: c:\users\sasch\documents\github\gecko-dev\xpcom\idl-parser\xpidl\xpidl.py:1647: Invalid regular expression for rule 't_singlelinecomment'. global flags not at the start of the expression at position 24
 0:47.58 ERROR: c:\users\sasch\documents\github\gecko-dev\xpcom\idl-parser\xpidl\xpidl.py:1660: Invalid regular expression for rule 't_LCDATA'. global flags not at the start of the expression at position 13

For example:

    def t_singlelinecomment(self, t):
        r"(?m)//.*?$"

This is correct as-is, but then lexer interpolates it within another string:

c = re.compile('(?P<%s>%s)' % (fname, _get_regex(f)), self.reflags)

Now suddenly (?m) appears at the middle of the regex expression, which is now illegal in Python 3.11. We should find a way to pass this flag as self.reflags instead of inline flags.

Note that this has been deprecated since 2016: https://github.com/python/cpython/commit/db99e6ee57c5070e6ef82fd2eaa528f09f3948dc

I don't think ply supports reflags for each syntax, it just accepts initially via lex.lex(reflags=reflags) and that's it. https://www.dabeaz.com/ply/ply.html

Looks quite tricky to fix, we might need to customize the library (if then it shouldn't be difficult IMO).

The product::component has been changed since the backlog priority was decided, so we're resetting it.
For more information, please visit auto_nag documentation.

(May be relevant if this ends up patching ply)