When I do an Android device build and try to run |mach run| or |mach mochitest|, I get the following error: $ ./mach run Error running mach: ['run'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: ADBError: ADBDevice.__init__: ls not found File "/home/botond/dev/mozilla/refactoring/python/mozbuild/mozbuild/mach_commands.py", line 886, in run verify_android_device(self, install=True) File "/home/botond/dev/mozilla/refactoring/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 237, in verify_android_device device = _get_device(build_obj.substs, device_serial) File "/home/botond/dev/mozilla/refactoring/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 101, in _get_device device = ADBAndroid(adb=adb_path, verbose=verbose_logging, device=device_serial) File "/home/botond/dev/mozilla/refactoring/testing/mozbase/mozdevice/mozdevice/adb_android.py", line 96, in __init__ require_root=require_root) File "/home/botond/dev/mozilla/refactoring/testing/mozbase/mozdevice/mozdevice/adb.py", line 653, in __init__ raise ADBError("ADBDevice.__init__: ls not found") Any suggestions on how to fix this, or how to investigate further? This is significantly impairing my ability to debug Fennec/GeckoView bugs.

What type of phone is this? What version of Android? Can you find 'ls' on the phone? We expect it at /system/bin/ls or /system/xbin/ls.

(In reply to Geoff Brown [:gbrown] from comment #1) > What type of phone is this? What version of Android? It's a Sony Xperia Z3C, running a stock version of Android 6.0.1. > Can you find 'ls' on the phone? We expect it at /system/bin/ls or /system/xbin/ls. /system/bin/ls is present and seems to functioning.

That code tries to run /system/bin/ls /data/local/tmp If /system/bin/ls is executable, maybe /data/local/tmp is not accessible? Does /data/local exist? Does /data/local/tmp exist? Can they be read?

from mozilla source dir export PYTHONPATH=$(pwd)/testing/mozbase/mozdevice:$(pwd)/testing/mozbase/mozfile:$(pwd)/testing/mozbase start python interpreter import logging from mozdevice import ADBAndroid logging.basicConfig() logger = logging.getLogger('root') logger.setLevel(logging.DEBUG) device = ADBAndroid(verbose=True, logger_name='root') paste the output as an attachment. Are you on macOS or Linux? Version?

Also include the output from device = ADBAndroid(verbose=True, logger_name='root', require_root=False) please. Thanks.

(In reply to Geoff Brown [:gbrown] from comment #3) > That code tries to run > > /system/bin/ls /data/local/tmp > > If /system/bin/ls is executable, maybe /data/local/tmp is not accessible? > Does /data/local exist? Does /data/local/tmp exist? Can they be read? It looks like /data/local/tmp is accessible, but /data/local isn't: $ adb shell /system/bin/ls /data/local opendir failed, Permission denied $ adb shell /system/bin/ls /data/local/tmp $ Could that be the problem?

(In reply to Bob Clary [:bc:] from comment #4) > from mozilla source dir > > export > PYTHONPATH=$(pwd)/testing/mozbase/mozdevice:$(pwd)/testing/mozbase/mozfile: > $(pwd)/testing/mozbase > > start python interpreter > > import logging > from mozdevice import ADBAndroid > logging.basicConfig() > logger = logging.getLogger('root') > logger.setLevel(logging.DEBUG) > device = ADBAndroid(verbose=True, logger_name='root') > > paste the output as an attachment. $ export PYTHONPATH=$(pwd)/testing/mozbase/mozdevice:$(pwd)/testing/mozbase/mozfile:$(pwd)/testing/mozbase $ python Python 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import logging >>> from mozdevice import ADBAndroid >>> logging.basicConfig() >>> logger = logging.getLogger('root') >>> logger.setLevel(logging.DEBUG) >>> device = ADBAndroid(verbose=True, logger_name='root') DEBUG:root:ADBAndroid: {'_require_root': True, '_logger': <logging.Logger object at 0x7f2615bc3d10>, '_adb_version': '', '_adb_host': None, '_adb_port': None, '_timeout': 300, '_polling_interval': 0.1, '_adb_path': 'adb', '_verbose': True} INFO:root:Using adb 1.0.40 DEBUG:root:shell_output: adb -s YT910VTCUK wait-for-device shell id; echo rc=$?, timeout: 300, root: False, timedout: None, exitcode: 0, output: uid=2000(shell) gid=2000(shell) groups=2000(shell),1004(input),1007(log),1011(adb),1015(sdcard_rw),1026(drmrpc),1028(sdcard_r),2993(trimarea),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats) context=u:r:shell:s0 DEBUG:root:command_output: adb -s YT910VTCUK wait-for-device root, timeout: 300, timedout: None, exitcode: 0, output: restarting adbd as root DEBUG:root:shell_output: adb -s YT910VTCUK wait-for-device shell id; echo rc=$?, timeout: 300, root: False, timedout: None, exitcode: 0, output: uid=0(root) gid=0(root) groups=0(root),1004(input),1007(log),1011(adb),1015(sdcard_rw),1026(drmrpc),1028(sdcard_r),2993(trimarea),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats) context=u:r:shell:s0 INFO:root:adbd running as root INFO:root:adbd restarted as root DEBUG:root:Check for su -c failed DEBUG:root:Check for su 0 failed Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/botond/dev/mozilla/refactoring/testing/mozbase/mozdevice/mozdevice/adb_android.py", line 96, in __init__ require_root=require_root) File "/home/botond/dev/mozilla/refactoring/testing/mozbase/mozdevice/mozdevice/adb.py", line 653, in __init__ raise ADBError("ADBDevice.__init__: ls not found") mozdevice.adb.ADBError: ADBDevice.__init__: ls not found >>> device = ADBAndroid(verbose=True, logger_name='root', require_root=False) DEBUG:root:ADBAndroid: {'_require_root': False, '_logger': <logging.Logger object at 0x7f2615bc3d10>, '_adb_version': '', '_adb_host': None, '_adb_port': None, '_timeout': 300, '_polling_interval': 0.1, '_adb_path': 'adb', '_verbose': True} INFO:root:Using adb 1.0.40 DEBUG:root:shell_output: adb -s YT910VTCUK wait-for-device shell id; echo rc=$?, timeout: 300, root: False, timedout: None, exitcode: 0, output: uid=0(root) gid=0(root) groups=0(root),1004(input),1007(log),1011(adb),1015(sdcard_rw),1026(drmrpc),1028(sdcard_r),2993(trimarea),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats) context=u:r:shell:s0 INFO:root:adbd running as root Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/botond/dev/mozilla/refactoring/testing/mozbase/mozdevice/mozdevice/adb_android.py", line 96, in __init__ require_root=require_root) File "/home/botond/dev/mozilla/refactoring/testing/mozbase/mozdevice/mozdevice/adb.py", line 653, in __init__ raise ADBError("ADBDevice.__init__: ls not found") mozdevice.adb.ADBError: ADBDevice.__init__: ls not found >>> > Are you on macOS or Linux? Version? Linux, Debian stable.

(In reply to Bob Clary [:bc:] from comment #5) > Also include the output from device = ADBAndroid(verbose=True, > logger_name='root', require_root=False) please. Thanks. (Included at the end of the previous output.)

I suppose we could try a different path though there might be other lurking issues we would need to find/resolve.

I believe /data/local/tmp was a hold over from the old days when devices had physical sdcards which might or might not be physically present. egao: Would you like to take a stab at this? I think testing /system would be sufficient. It should be present and accessible on any device. It would be nice if the error message was more accurate rather than implying that ls was not present. We would want to make sure that the local mach commands continue to work, that the unit tests continue to work and that mozregression also continues to work. You can get a picture of the steps for working with mozregression from Bug 1484607. wlach can help with anything related to mozregression and bogdan.surd@softvision.ro can help with testing the various devices he has available as well. gbrown: thoughts?

Changing to /system seems like a great idea. :botond -- This problem is related to the file permissions on your device. You can very likely run 'mach run' or 'mach mochitest' on an emulator without any trouble, while we sort this out. (Unplug your device, run your mach command, and let mach setup an emulator for you.)

(In reply to Geoff Brown [:gbrown] from comment #11) > You can very likely run 'mach run' or 'mach mochitest' on an emulator > without any trouble, while we sort this out. (Unplug your device, run your > mach command, and let mach setup an emulator for you.) I tried this and it sort of worked, although the emulator it _really_ slow, and I'm finding scrolling isn't very reliable in it when you're trying to interact wit it (possibly related to the slowness?). It does seem to run a mochitest fine though, so that helps. |mach run --debug| doesn't work, but I understand that might currently be broken on devices too.

Extra try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=84c10b38b3f46c21d39ae687d1badd249fb81774 ---- The only changes that were made was to check a different directory. I had tried out using 'stat' to check the presence of ls binary in each of the respective directories, but that caused failures at inexplicable locations.

- changed the directory for ls to check to /system.

Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/476c7c57b03c change how ADBAndroid checks for presence of 'ls' command r=bc,gbrown

:botond - Would you try applying this patch and trying to run/mochitest on your device? Does it help?

Yup, I can confirm that I get past this error now. Thanks for the fix!

While |mach run| and and |mach mochitest| are working now, I still get the same error when trying to use mozregression on my Android device. Is there a similar fix that needs to be made to mozregression?

mozregression uses an old version of mozdevice. I assume mozregression installed it globally which in case you can sudo pip install mozdevice --upgrade to pick up 1.1.5.

(In reply to Bob Clary [:bc:] from comment #20) > mozregression uses an old version of mozdevice. I assume mozregression > installed it globally which in case you can sudo pip install mozdevice > --upgrade to pick up 1.1.5. Thanks. When I do that I get: mozregression 2.3.34 has requirement mozdevice==1.1.1, but you'll have mozdevice 1.1.5 which is incompatible. but it looks like it updated it anyways? I filed bug 1503712 for updating the requirement. However, mozregression is still not working: it restarts adb as root, and then tries to install the apk, and on my phone that fails with a permission error. Should I file a separate mozregression bug about this?

(In reply to Botond Ballo [:botond] from comment #21) > > However, mozregression is still not working: it restarts adb as root, and > then tries to install the apk, and on my phone that fails with a permission > error. Should I file a separate mozregression bug about this? This sounds more like a mozdevice bug than a mozregression one. This seems like strange behaviour though, I know :bc has gone to some effort recently to make sure mozdevice doesn't require root for basic operations like installing apks

I don't have time to look into mozregression any time soon. If no one else gets to it first, I'll take a look when I get some of my other tasks completed.

(In reply to Botond Ballo [:botond] from comment #21) > However, mozregression is still not working: it restarts adb as root, and > then tries to install the apk, and on my phone that fails with a permission > error. Should I file a separate mozregression bug about this? Maybe just mention mozregression in bug 1501867; I suspect it's all the same issue.

(In reply to Geoff Brown [:gbrown] from comment #24) > (In reply to Botond Ballo [:botond] from comment #21) > > However, mozregression is still not working: it restarts adb as root, and > > then tries to install the apk, and on my phone that fails with a permission > > error. Should I file a separate mozregression bug about this? > > Maybe just mention mozregression in bug 1501867; I suspect it's all the same > issue. Mentioned. Note, though, that while bug 1501867 is merely an annoyance (I have to run |adb unroot| from time to time), the issue with mozregression is actually blocking me from using mozregression. If anyone has a suggested workaround for that, that would be greatly appreciated.

The issue in this bug is fixed. Let's move whatever the issue you are having to a new bug. I suspect this is the same as Bug 1500653. Please go there and answer the questions in Bug 1500653 comment 2.

(In reply to Bob Clary [:bc:] from comment #26) > The issue in this bug is fixed. Let's move whatever the issue you are having > to a new bug. I suspect this is the same as Bug 1500653. Please go there > and answer the questions in Bug 1500653 comment 2. I filed a new bug 1504259, as the symptoms seemed different (not being able to install the apk vs. the apk crashing after being installed). I did answer the same questions, in bug 1504259 comment 3.