Currently, to create a mach command, you must do:

@CommandProvider
class CustomCommandClass(MachCommandBase):
    @Command(...)
    @CommandArgument(...)
    @CommandArgument(...)
    def my_command(self, argument1, argument2):
        # Command arguments are arguments to this function,
        # context information is stored as member variables of MachCommandBzse
        print("That was a lot of ceremony!")

The downsides of this are:

• There's more ceremony in the code (what is this, Java?)
• Adding an __init__(...) requires that you bubble parameters up to MachCommandBase.
• It requires the creation of MachCommandBase to test mach commands - it's hard to "mock out" parent classes.

Instead, we should allow defining commands as:

@Command(...)
@CommandArgument(...)
@CommandArgument(...)
def my_command(mach_context, argument1, argument2):
    # What used to be stored in `self` is now in `mach_context`.
    # Arguments are still provided as function parameters
    print("This was so convenient, mach is incredible software")

EDIT: See comment 13 for more context.

This gives the Command and SubCommand decorators the capability to register
commands specified as standalone functions, while retaining the compatibility
with classes as before.

In preparation for future changes in how these decorators work,
add a few basic tests.

As a step to remove the need to use @CommandProvider, we are moving
the metrics_path parameter to @Command.

Depends on D107836

https://hg.mozilla.org/mozilla-central/rev/3e834be2046b

Still more work to do yet :)

https://hg.mozilla.org/mozilla-central/rev/a8dcd8e64a85

As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).

Depends on D109650

Depends on D110777

As a step towards moving mach commands outside of classes, this converts all
properties into methods so that they can later become top-level helper functions.

Depends on D109650

Some context for teams tuning into this bug because a Mach command they maintain is being updated by Alex:

All of our Mach commands right now are stored as functions in classes, but the class abstraction is unnecessary - all the methods of such a CommandProvider can be happily moved up to the class-level, simplifying the code and making future mach commands more easy to write.

However, to achieve this, we need to make some tweaks:

• Modules can't have @property functions (of course), so we're turning properties into regular helper functions.
• Modules can't (well, shouldn't) have initialization logic, so pieces from __init__() are being moved into helper functions.
• There will no longer be a self, so we're replacing that with a command_context variable that has the same goodies.
• Then, at the end, we can remove the top-level class and shuffle everything up a level.

We're attempting to do these changes in a way that's both easy-to-review and keeps the entire tree consistent.
So, if you're surprised by some changes happening to your local Mach command, then they can probably be explained by ^ :)

Another step towards avoiding the need for classes in mach commands;
here we are removing constructors either by changing them into helpers
or by simple refactorings.

Depends on D112196

Backed out for non-unified build bustages.

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=d4a5d856797740630c86cb37dc57d14965b3f0e3&selectedTaskRun=XAF4G5OaQyGHQnGfmq05ag.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=337013895&repo=autoland

Backout link: https://hg.mozilla.org/integration/autoland/rev/b85edcc1ce08369f27947268f1ebef5787f8cbae

Hmm, looks like ./mach try auto didn't know all the affected commands.
Let's get those failed tasks in that "Push with failures" resolved, then we can land again.

Backed out for causing js-bench-sm failures.

Backout link: https://hg.mozilla.org/integration/autoland/rev/31a80696d4d5a3e745be3df2e86df061cf02daf5

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&selectedTaskRun=Mmc2pD0mQMGC5eW7lAM2Ew.0&revision=e7c3344e6ae7fd04cef466dd0efbd916a6929a45&searchStr=linux%2Cx64%2Cshippable%2Copt%2Cjavascript%2Cshell%2Cbenchmarks%2Cwith%2Cspidermonkey%2Csource-test-jsshell-bench-sunspider-sm%2Csunspider

Failures log: https://treeherder.mozilla.org/logviewer?job_id=337504457&repo=autoland&lineNumber=3264

https://hg.mozilla.org/mozilla-central/rev/e3f73f78f1c7

https://hg.mozilla.org/mozilla-central/rev/e949c39b93e6

https://hg.mozilla.org/mozilla-central/rev/eddb5c40cd92

Backed out for causing lint failures.

Backout link: https://hg.mozilla.org/integration/autoland/rev/ca386bcad42046b17d617e55a92e353ef2dc3061

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&searchStr=linting%2Copt%2Cpython%2Cchecks%2Csource-test-mozlint-py-black%2Cblack&revision=4b889750da57ff0994c7196bc8a0151c17f641b6&selectedTaskRun=OxD2lpNnRray3IVwO3Cn_w.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=341016479&repo=autoland&lineNumber=464

https://hg.mozilla.org/mozilla-central/rev/134e5702f160

This step removes all the dependencies of mach commands to
having a MachCommandBase as the self by using the command_context
argument instead. This also removes any remaining statefulness from those
classes that implement mach commands, ultimately making it easier to move
existing commands out of classes in a follow-up.

Backed out for causing bustages complaining about 'CommandContext'.

Backout link: https://hg.mozilla.org/integration/autoland/rev/abf91c0aeefdee7c7070dd9fb5f8895736d485ce

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=e1921c5112d83da21aa2fae7a9c5710016d1b371&selectedTaskRun=S08Dn7pYT6OMApnAu--kWA.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=345459952&repo=autoland&lineNumber=551
https://treeherder.mozilla.org/logviewer?job_id=345457500&repo=autoland&lineNumber=754

https://hg.mozilla.org/mozilla-central/rev/63279eccfdfc

When transitioning from using self to using command_context in mach
commands, this specific mistake went undetected (no bustage forced a backout,
either). This commit fixes it.

https://hg.mozilla.org/mozilla-central/rev/9075b050f763

This removes the @CommandProvider decorator and the need to implement
mach commands inside subclasses of MachCommandBase, and moves all
existing commands out from classes to module level functions.

The patch that made the replacement across all commands
missed this particular file.

https://hg.mozilla.org/mozilla-central/rev/ecfc828f27fc

This is to keep up with changes happening in commands while still transitioning
to commands without classes.

https://hg.mozilla.org/mozilla-central/rev/cacb7e90783b

The purpose of this is to remove as many docstrings from CommandProvider
classes to make the step of moving commands out of classes simpler.
Where possible, the docstring has been moved to or merged with the function.

https://hg.mozilla.org/mozilla-central/rev/343b5fd104a5

The affected functions would cause an UnboundLocalError if they are moved
to module-level, because using an assignment inside a function makes a
variable local in Python, making it illegal to have a local variable name
be the same as a function name that is called within the same function.
The adopted solution is to rename those functions in preparation for the
future de-classing.

https://hg.mozilla.org/mozilla-central/rev/3602cbaf8893

Backed out changeset 5f5b612878f3 (Bug 1696251) for causing multiple bustages

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&collapsedPushes=842382&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&searchStr=toolchains%2Copt%2Ctoolchain%2Cbuilds%2Cfor%2Clinux%2C64-bits%2Ctoolchain-browsertime%2Cbrowsertime&revision=5f5b612878f318b3e635faf5cc145155cd454eb0

Log: https://treeherder.mozilla.org/logviewer?job_id=351950916&repo=autoland&lineNumber=131
https://treeherder.mozilla.org/logviewer?job_id=351947200&repo=autoland&lineNumber=222
https://treeherder.mozilla.org/logviewer?job_id=351959226&repo=autoland&lineNumber=47661
Also, this perma tier2: https://treeherder.mozilla.org/logviewer?job_id=351943633&repo=autoland&lineNumber=50

Backout: https://hg.mozilla.org/integration/autoland/rev/e28c911d36db5524416becf1781c231a058e6c21

Backed out for causing bustages

• backout: https://hg.mozilla.org/integration/autoland/rev/b747274f1da3e93a8bed6b8c8df6299ceb354417

• push: https://treeherder.mozilla.org/jobs?repo=autoland&selectedTaskRun=E4bIIIdbQ4251DwWnwUC_w.0&revision=510dd46a9de764e8730e081235a5d4d3aa9507ee

• failure log: https://treeherder.mozilla.org/logviewer?job_id=352178753&repo=autoland&lineNumber=47886

[task 2021-09-21T01:44:34.588Z] 01:44:34     INFO -  01:44:34     INFO -  List of locales does not include default locale "en-US": ['an', 'ar', 'ast', 'az', 'be', 'bg', 'bn', 'br', 'bs', 'ca', 'cak', 'cs', 'cy', 'da', 'de', 'dsb', 'el', 'en-CA', 'en-GB', 'eo', 'es-AR', 'es-CL', 'es-ES', 'es-MX', 'et', 'eu', 'fa', 'ff', 'fi', 'fr', 'fy-NL', 'ga-IE', 'gd', 'gl', 'gn', 'gu-IN', 'he', 'hi-IN', 'hr', 'hsb', 'hu', 'hy-AM', 'id', 'is', 'it', 'ja', 'ka', 'kab', 'kk', 'kn', 'ko', 'lij', 'lo', 'lt', 'lv', 'ml', 'mr', 'ms', 'my', 'nb-NO', 'ne-NP', 'nl', 'nn-NO', 'oc', 'pa-IN', 'pl', 'pt-BR', 'pt-PT', 'rm', 'ro', 'ru', 'sk', 'sl', 'son', 'sq', 'sr', 'sv-SE', 'ta', 'te', 'th', 'tr', 'trs', 'uk', 'ur', 'uz', 'vi', 'wo', 'xh', 'zam', 'zh-CN', 'zh-TW']; adding "en-US"
[task 2021-09-21T01:44:34.590Z] 01:44:34     INFO -  01:44:34     INFO -  Error running mach:
[task 2021-09-21T01:44:34.591Z] 01:44:34     INFO -  01:44:34     INFO -      ['--log-no-times', 'package-multi-locale', '--locales', 'an', 'ar', 'ast', 'az', 'be', 'bg', 'bn', 'br', 'bs', 'ca', 'cak', 'cs', 'cy', 'da', 'de', 'dsb', 'el', 'en-CA', 'en-GB', 'eo', 'es-AR', 'es-CL', 'es-ES', 'es-MX', 'et', 'eu', 'fa', 'ff', 'fi', 'fr', 'fy-NL', 'ga-IE', 'gd', 'gl', 'gn', 'gu-IN', 'he', 'hi-IN', 'hr', 'hsb', 'hu', 'hy-AM', 'id', 'is', 'it', 'ja', 'ka', 'kab', 'kk', 'kn', 'ko', 'lij', 'lo', 'lt', 'lv', 'ml', 'mr', 'ms', 'my', 'nb-NO', 'ne-NP', 'nl', 'nn-NO', 'oc', 'pa-IN', 'pl', 'pt-BR', 'pt-PT', 'rm', 'ro', 'ru', 'sk', 'sl', 'son', 'sq', 'sr', 'sv-SE', 'ta', 'te', 'th', 'tr', 'trs', 'uk', 'ur', 'uz', 'vi', 'wo', 'xh', 'zam', 'zh-CN', 'zh-TW']
[task 2021-09-21T01:44:34.591Z] 01:44:34     INFO -  01:44:34     INFO -  The error occurred in the implementation of the invoked mach command.
[task 2021-09-21T01:44:34.591Z] 01:44:34     INFO -  01:44:34     INFO -  This should never occur and is likely a bug in the implementation of that
[task 2021-09-21T01:44:34.591Z] 01:44:34     INFO -  01:44:34     INFO -  command.
[task 2021-09-21T01:44:34.591Z] 01:44:34     INFO -  01:44:34     INFO -  You can invoke |./mach busted| to check if this issue is already on file. If it
[task 2021-09-21T01:44:34.591Z] 01:44:34     INFO -  01:44:34     INFO -  isn't, please use |./mach busted file package-multi-locale| to report it. If |./mach busted| is
[task 2021-09-21T01:44:34.591Z] 01:44:34     INFO -  01:44:34     INFO -  misbehaving, you can also inspect the dependencies of bug 1543241.
[task 2021-09-21T01:44:34.591Z] 01:44:34     INFO -  01:44:34     INFO -  If filing a bug, please include the full output of mach, including this error
[task 2021-09-21T01:44:34.591Z] 01:44:34     INFO -  01:44:34     INFO -  message.
[task 2021-09-21T01:44:34.591Z] 01:44:34     INFO -  01:44:34     INFO -  The details of the failure are as follows:
[task 2021-09-21T01:44:34.591Z] 01:44:34     INFO -  01:44:34     INFO -  AttributeError: 'list' object has no attribute '_get_state_filename'
[task 2021-09-21T01:44:34.592Z] 01:44:34     INFO -  01:44:34     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/mach_commands.py", line 2281, in package_l10n
[task 2021-09-21T01:44:34.592Z] 01:44:34     INFO -  01:44:34     INFO -      locales = list(sorted(locales))
[task 2021-09-21T01:44:34.592Z] 01:44:34     INFO -  01:44:34     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/mach_commands.py", line 476, in list
[task 2021-09-21T01:44:34.592Z] 01:44:34     INFO -  01:44:34     INFO -      database = get_warnings_database(command_context)
[task 2021-09-21T01:44:34.592Z] 01:44:34     INFO -  01:44:34     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/mach_commands.py", line 407, in get_warnings_database
[task 2021-09-21T01:44:34.592Z] 01:44:34     INFO -  01:44:34     INFO -      path = database_path(command_context)
[task 2021-09-21T01:44:34.592Z] 01:44:34     INFO -  01:44:34     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/mach_commands.py", line 401, in database_path
[task 2021-09-21T01:44:34.592Z] 01:44:34     INFO -  01:44:34     INFO -      return command_context._get_state_filename("warnings.json")
[task 2021-09-21T01:44:34.635Z] 01:44:34     INFO -  01:44:34    ERROR - Return code: 1
[task 2021-09-21T01:44:34.635Z] 01:44:34     INFO -  01:44:34    FATAL - 'mach package-multi-locale --locales an ar ast az be bg bn br bs ca cak cs cy da de dsb el en-CA en-GB eo es-AR es-CL es-ES es-MX et eu fa ff fi fr fy-NL ga-IE gd gl gn gu-IN he hi-IN hr hsb hu hy-AM id is it ja ka kab kk kn ko lij lo lt lv ml mr ms my nb-NO ne-NP nl nn-NO oc pa-IN pl pt-BR pt-PT rm ro ru sk sl son sq sr sv-SE ta te th tr trs uk ur uz vi wo xh zam zh-CN zh-TW' did not run successfully. Please check log for errors.
[task 2021-09-21T01:44:34.635Z] 01:44:34     INFO -  01:44:34    FATAL - Running post_fatal callback...
[task 2021-09-21T01:44:34.635Z] 01:44:34     INFO -  01:44:34    FATAL - Exiting -1

Please make sure to address bug 1731703 before relanding (which was also the error in the second log in comment 54).

https://hg.mozilla.org/mozilla-central/rev/f25fa6972142

https://hg.mozilla.org/mozilla-central/rev/803ef9f68ea8
https://hg.mozilla.org/mozilla-central/rev/53b1fa0faa6d

Fantastic work Alex! The widespread refactor is complete 😅

I'm going to broadcast this a bit later today and touch on its main benefits:

• Less boilerplate/complexity when defining Mach commands
• Mach commands are far easier to test, because you no longer need to do a parent-class monkeypatch to instrument your command without having unrelated MozbuildObject/MachCommandBase work going on (parsing mozconfig, etc).

Backed out for breaking the static-analysis integration: https://hg.mozilla.org/mozilla-central/rev/c38d34be7c3f757510f037fef5a78ea08af32980

In this patch, we differentiate methods that actually come from the Build
class from those that actually come from the parent MachCommandBase class,
which will have to be called on the command_context object. Also, we rename
builder to build_commands in preparation for those methods becoming top-level
functions of the module of the same name.

https://hg.mozilla.org/mozilla-central/rev/5c95fce39b9c
https://hg.mozilla.org/mozilla-central/rev/94ababc37c90