$ hg amend
Traceback (most recent call last):
  File "path\to\mozilla-central\python/mozlint\mozlint\roller.py", line 39, in _run_worker
    res = func(paths, config, **lintargs) or []
  File "path\to\mozilla-central\python/mozlint\mozlint\types.py", line 50, in __call__
    return self._lint(paths, config, **lintargs)
  File "path\to\mozilla-central\python/mozlint\mozlint\types.py", line 133, in _lint
    func = findobject(config['payload'])
  File "path\to\mozilla-central\python/mozlint\mozlint\pathutils.py", line 216, in findobject
    obj = __import__(modulepath)
  File "path\to\mozilla-central\tools\lint\eslint\__init__.py", line 16, in <module>
    from eslint import setup_helper
  File "path\to\mozilla-central\tools\lint\eslint\setup_helper.py", line 20, in <module>
    from mozbuild.nodeutil import (find_node_executable, find_npm_executable,
  File "path\to\mozilla-central\python/mozbuild\mozbuild\nodeutil.py", line 15, in <module>
    from mozfile import which
ImportError: cannot import name which
A failure occurred in the eslint linter.

When the commit hook is configured through:

[hooks]
pretxncommit.lint = python:/mozilla-central/tools/lint/hooks.py:hg

on a commit based on https://hg.mozilla.org/mozilla-central/rev/db8f3ee41bdf .

I tried running ./mach bootstrap to no avail.

./mach lint --outgoing produces the same error.

Clearing the .pyc files inside testing/mozbase/mozfile/mozfile/ doesn't help, and --verbose doesn't provide more output.

In case the mixed slashes in the path didn't make it obvious, this is on Windows.

Thanks for filing, would you mind applying this patch letting me know what the printed path is?

diff --git a/python/mozbuild/mozbuild/nodeutil.py b/python/mozbuild/mozbuild/nodeutil.py
--- a/python/mozbuild/mozbuild/nodeutil.py
+++ b/python/mozbuild/mozbuild/nodeutil.py
@@ -7,16 +7,18 @@ from __future__ import absolute_import, 
 import os
 import subprocess
 import platform
 from distutils.version import (
     StrictVersion,
 )
 
 from mozboot.util import get_state_dir
+import mozfile
+print(mozfile.__file__)
 from mozfile import which
 
 NODE_MIN_VERSION = StrictVersion("8.11.0")
 NPM_MIN_VERSION = StrictVersion("5.6.0")
 
 
 def find_node_paths():
     """ Determines the possible paths for node executables.

(I think clobbering <objdir>/_virtualenvs might fix it)

(In reply to Andrew Halberstadt [:ahal] from comment #2)

Thanks for filing, would you mind applying this patch letting me know what the printed path is?

It prints twice:

path\to\mozilla-central\testing/mozbase/mozfile\mozfile\__init__.pyc
path\to\opt-objdir\_virtualenvs\init\Lib\site-packages\mozfile\__init__.pyc

So looks like you're probably right? At what point should such a clobber normally happen (that is, I assume that something normally takes care of this automatically?)

I can confirm that moving the $OBJDIR/_virtualenvs file to an old_* dir fixes the issue...

I don't think there's any automatic clobbering mechanism. I'm not an expert on our virtualenv creation but I did a bit of digging and the reason this type of error doesn't happen all the time is:

1. Only mozbase modules are placed in the init virtualenv, all other modules (e.g in third_party/python) are appended to the PYTHONPATH at mach runtime. I'm not sure why this is, probably something to do with mach or the virtualenv logic depending on mozbase.

2. On non-Windows, these mozbase modules link back to the mozbase source.

Together that means this type of issue should only happen on Windows and with changes to mozbase, which changes pretty infrequently these days. Maybe one easy thing to do would be to change mach clobber python to also remove the _virtualenvs folder (in addition to removing all .pyc files). That way at least we can tell people to run a simple one-liner if they run into this.

(In reply to Andrew Halberstadt [:ahal] from comment #6)

I don't think there's any automatic clobbering mechanism. I'm not an expert on our virtualenv creation but I did a bit of digging and the reason this type of error doesn't happen all the time is:

1. Only mozbase modules are placed in the init virtualenv, all other modules (e.g in third_party/python) are appended to the PYTHONPATH at mach runtime. I'm not sure why this is, probably something to do with mach or the virtualenv logic depending on mozbase.

2. On non-Windows, these mozbase modules link back to the mozbase source.

Together that means this type of issue should only happen on Windows and with changes to mozbase, which changes pretty infrequently these days. Maybe one easy thing to do would be to change mach clobber python to also remove the _virtualenvs folder (in addition to removing all .pyc files). That way at least we can tell people to run a simple one-liner if they run into this.

FWIW, I just saw the same thing on mac, but I suspect that's due to bug 1549186...

I also ran into this on Windows today, removing $OBJDIR/_virtualenvs did the trick.

I seem to have found another way to reproduce that is... unfortunate:

1. set up mozconfig with clean objdir, build
2. run ./mach mozregression --help, which will set up mozregression and installs a bunch of stuff in the virtualenv
3. touch some cpp file and run ./mach build binaries

Fortunately, deleting the virtualenv fixes this and "only" requires another configure step plus a minor amount of additional build work, rather than a full-on clobber. But it's unfortunate.

Can we make ./mach mozregression not break stuff like this?

I've actually never touched mozregression before, but yeah.. we could probably give it its own dedicated virtualenv and it could use a Pipfile to specify whatever packages it needs. Similar to this:
https://searchfox.org/mozilla-central/source/tools/docs/mach_commands.py#65

When I ran ./mach mozregression locally it didn't seem to install anything (even after clobbering).. but maybe that's because I'm on Linux? Might be good to file a bug under Testing :: mozregression and cc :wlach (I'm not sure if he's still the owner or not, but he can at least direct you to whoever is).

OK, filed bug 1580625 for the mozregression thingy.

I am pretty sure bug 1582248 should workaround this particular issue, though it's admittedly only a short term solution. If it works as I hope, maybe we can mark this as a dupe? Bug 1580625 remains open for a more permanent fix.

I guess we could have just continued using this bug rather than filing bug 1580625.. but oh well. Since most of the mozregression talk is over there let's use that to solve properly.