Closed Bug 1547730 Opened 6 years ago Closed 5 years ago

Make much of mozbuild/mozbuild and mozbuild/mozpack py3 compatible, ensuring mozpack.mozjar can run on py3

Categories

(Firefox Build System :: General, enhancement)

enhancement
Not set
normal

Tracking

(firefox69 fixed)

RESOLVED FIXED
mozilla69
Tracking Status
firefox69 --- fixed

People

(Reporter: Callek, Assigned: Callek)

References

Details

Attachments

(28 files, 2 obsolete files)

(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
No description provided.
Type: defect → enhancement

This is necessary for any attempt at @import('six') or doing any similar py2+py3 work
in the configure sandbox.

shellutil.py needed special handling since it is invoked earlier than the
vendor directory adding we do, so it does not have six available.

(adds build_class to the builtin sandbox and allow CombinedDependsFunction to be hashed)

Attached file Bug 1547730 - Use six.moves for urlparse r=#build (obsolete) (deleted) —
Attachment #9062099 - Attachment is obsolete: true
Attachment #9062098 - Attachment is obsolete: true
Attachment #9062093 - Attachment description: Bug 1547730 - Define __hash__ for SandboxDependsFunction so we can reference it as a dictionary key in py3 r=#build → Bug 1547730 - Define __hash__ for SandboxDependsFunction and CombinedDependsFunction so we can reference it as a dictionary key in py3 r=#build
Pushed by Callek@gmail.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/b4e024474194 Add six to path when running configure. r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/afa5801534e4 Do dict.iteritems() via six to support py3. r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/81bc9014907e Do six.itervalues instead of dict.itervalues r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/042cdcc4e103 use six.iterkeys() r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/c846bf431b5c Allow testing of mozpack on py3 r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/1b766ee6bf23 Switch to six.string_types instead of types.StringTypes for py3 r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/35e2dc6ad347 Do s/StandardError/Exception/ to support py3 r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/fa39fb43fd12 do py3 for cStringIO r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/aa2acfd554fc do s/StringIO.StringIO/six.StringIO/ to support py3 r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/b7b1062d0aad don't use py2 builtin 'unicode' in mozpack and deps r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/16b1c53aba9d Don't try to py3-ize indented_repr yet. r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/1891cfbb55d1 Compensate for __builtin__ use in files accessed by py3 r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/3df74540bde3 Use six.moves.xrange() instead of xrange() r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/3037f5bf1019 Fix b''.join(rand.choice(bytestring)..) to use bytes() constructor instead r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/6c9f98dacbde Use six.moves for urllib.urlparse r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/03bdcaab1623 Use six.moves.reduce for reduce() r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/75879e53727c Recognize StreamWriter exceptions for fileno() in py3 r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/2d6ba80390c8 Stop using some cases of types.NoneType. r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/61b8fcc639e0 Define __hash__ for SandboxDependsFunction and CombinedDependsFunction so we can reference it as a dictionary key in py3 r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/59a1393697e0 Use new inspection methods introduced in py3 but work in py2.7 for functions r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/cef492de6a7f Use six.moves.builtins in the sandbox for configure r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/48f2c4980ad0 Make more of configure/__init__ work in py3 r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/27ff9602d80a Update a build moz.configure function, to support py3 r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/3df19a96f8ac Finish up some final py3-izing in getting mozjar.py to work in py3 r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/d282ddabcb3d Set python test annotations to correspond to currently-passing py3 tests r=#build https://hg.mozilla.org/integration/mozilla-inbound/rev/9e9c2e06d0fb don't fail on an unsupported .seek() needed to support adding from compressed `zipfile.ZipFile` entries. r=#build

This is due to a conflicting name in sys path when called from mozharness,
Which we fix by moving the codecoverage.py file in mozharness to codecov.py
avoiding pythons confusion.

Depends on D32407

Pushed by jwood@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/198001a673f6 Add six to path when running configure. r=glandium https://hg.mozilla.org/integration/autoland/rev/abbfe68afc04 Do dict.iteritems() via six to support py3. r=glandium https://hg.mozilla.org/integration/autoland/rev/66933913a2df Do six.itervalues instead of dict.itervalues r=glandium https://hg.mozilla.org/integration/autoland/rev/1e4407c3c9e3 use six.iterkeys() r=glandium https://hg.mozilla.org/integration/autoland/rev/5588a24fd92c Allow testing of mozpack on py3 r=glandium https://hg.mozilla.org/integration/autoland/rev/01397bf93fc4 Switch to six.string_types instead of types.StringTypes for py3 r=glandium https://hg.mozilla.org/integration/autoland/rev/6c6c8728e836 Do s/StandardError/Exception/ to support py3 r=glandium https://hg.mozilla.org/integration/autoland/rev/228da077174e do py3 for cStringIO r=glandium https://hg.mozilla.org/integration/autoland/rev/af2c8864114c do s/StringIO.StringIO/six.StringIO/ to support py3 r=glandium https://hg.mozilla.org/integration/autoland/rev/f6509e01bf77 don't use py2 builtin 'unicode' in mozpack and deps r=glandium https://hg.mozilla.org/integration/autoland/rev/47e737a70bea Don't try to py3-ize indented_repr yet. r=glandium https://hg.mozilla.org/integration/autoland/rev/d74097a6c6c5 Compensate for __builtin__ use in files accessed by py3 r=glandium https://hg.mozilla.org/integration/autoland/rev/2dafc77a00b2 Use six.moves.xrange() instead of xrange() r=glandium https://hg.mozilla.org/integration/autoland/rev/5839ef6f14c5 Fix b''.join(rand.choice(bytestring)..) to use bytes() constructor instead r=glandium https://hg.mozilla.org/integration/autoland/rev/f234dadb03c2 Use six.moves for urllib.urlparse r=glandium https://hg.mozilla.org/integration/autoland/rev/5eefa13f42a6 Use six.moves.reduce for reduce() r=glandium https://hg.mozilla.org/integration/autoland/rev/7d416b0e2d4e Recognize StreamWriter exceptions for fileno() in py3 r=glandium https://hg.mozilla.org/integration/autoland/rev/577509d87235 Stop using some cases of types.NoneType. r=glandium https://hg.mozilla.org/integration/autoland/rev/bd44eb6beabb Define __hash__ for SandboxDependsFunction and CombinedDependsFunction so we can reference it as a dictionary key in py3 r=glandium https://hg.mozilla.org/integration/autoland/rev/d89d2754488e Use new inspection methods introduced in py3 but work in py2.7 for functions r=glandium https://hg.mozilla.org/integration/autoland/rev/e7cfc61f2bc3 Use six.moves.builtins in the sandbox for configure r=glandium https://hg.mozilla.org/integration/autoland/rev/690a6fec7137 Make more of configure/__init__ work in py3 r=glandium https://hg.mozilla.org/integration/autoland/rev/29daeec3a135 Update a build moz.configure function, to support py3 r=glandium https://hg.mozilla.org/integration/autoland/rev/a3d287588b12 Finish up some final py3-izing in getting mozjar.py to work in py3 r=glandium https://hg.mozilla.org/integration/autoland/rev/ef2589543f46 Set python test annotations to correspond to currently-passing py3 tests r=glandium https://hg.mozilla.org/integration/autoland/rev/7d0b13ac76f2 don't fail on an unsupported .seek() needed to support adding from compressed `zipfile.ZipFile` entries. r=glandium https://hg.mozilla.org/integration/autoland/rev/12061a0decdb Fix precomplete file generation when passing in unicode strings. r=glandium https://hg.mozilla.org/integration/autoland/rev/dbadda4e3c1c Fix use of absolute imports in codecoverage/*.py r=tomprince
Flags: needinfo?(bugspam.Callek)

https://hg.mozilla.org/mozilla-central/rev/198001a673f6
https://hg.mozilla.org/mozilla-central/rev/abbfe68afc04
https://hg.mozilla.org/mozilla-central/rev/66933913a2df
https://hg.mozilla.org/mozilla-central/rev/1e4407c3c9e3
https://hg.mozilla.org/mozilla-central/rev/5588a24fd92c
https://hg.mozilla.org/mozilla-central/rev/01397bf93fc4
https://hg.mozilla.org/mozilla-central/rev/6c6c8728e836
https://hg.mozilla.org/mozilla-central/rev/228da077174e
https://hg.mozilla.org/mozilla-central/rev/af2c8864114c
https://hg.mozilla.org/mozilla-central/rev/f6509e01bf77
https://hg.mozilla.org/mozilla-central/rev/47e737a70bea
https://hg.mozilla.org/mozilla-central/rev/d74097a6c6c5
https://hg.mozilla.org/mozilla-central/rev/2dafc77a00b2
https://hg.mozilla.org/mozilla-central/rev/5839ef6f14c5
https://hg.mozilla.org/mozilla-central/rev/f234dadb03c2
https://hg.mozilla.org/mozilla-central/rev/5eefa13f42a6
https://hg.mozilla.org/mozilla-central/rev/7d416b0e2d4e
https://hg.mozilla.org/mozilla-central/rev/577509d87235
https://hg.mozilla.org/mozilla-central/rev/bd44eb6beabb
https://hg.mozilla.org/mozilla-central/rev/d89d2754488e
https://hg.mozilla.org/mozilla-central/rev/e7cfc61f2bc3
https://hg.mozilla.org/mozilla-central/rev/690a6fec7137
https://hg.mozilla.org/mozilla-central/rev/29daeec3a135
https://hg.mozilla.org/mozilla-central/rev/a3d287588b12
https://hg.mozilla.org/mozilla-central/rev/ef2589543f46
https://hg.mozilla.org/mozilla-central/rev/7d0b13ac76f2
https://hg.mozilla.org/mozilla-central/rev/12061a0decdb
https://hg.mozilla.org/mozilla-central/rev/dbadda4e3c1c

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla69
Regressions: 1555894
No longer regressions: 1555894
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: