xidorn: This might have been fixed by bug 1486004. Can you check if it is still reproducible?

It doesn't crash, but it doesn't run at all now. The output is:

 0:02.69 INFO: Using date 2018-09-05 for release 63
 0:05.79 INFO: Downloading build from: https://archive.mozilla.org/pub/mobile/nightly/2018/09/2018-09-05-10-01-17-mozilla-central-android-api-16/fennec-64.0a1.multi.android-arm.apk
===== Downloaded 100% =====
 0:11.83 INFO: Running mozilla-central build for 2018-09-05
 0:11.85 INFO: application_buildid: 20180905100117
 0:11.85 INFO: application_changeset: 26990836dc5cc3cd1b8027392b79210e71094dc3
 0:11.85 INFO: application_name: Fennec
 0:11.85 INFO: application_repository: https://hg.mozilla.org/mozilla-central
 0:11.85 INFO: application_version: 64.0a1
 0:11.85 INFO: package_name: org.mozilla.fennec_aurora
 0:11.86 INFO: Using adb 1.0.39
 0:12.37 INFO: /system/bin/ls -a supported
 0:12.48 INFO: Native cp support: True
 0:12.58 INFO: Native chmod -R support: True
 0:12.68 INFO: Native chown -R support: True
 0:12.99 INFO: Native normal pidof support: False
 0:13.40 INFO: adbd not restarted as root
 0:13.50 INFO: su -c setenforce 0 exitcode 0, stdout: None
 0:14.12 WARNING: Unable to set SELinux Permissive due to args: adb wait-for-device shell setenforce Permissive; echo adb_returncode=$?, exitcode: 1, stdout: setenforce: Couldn't set enforcing status to 'Permissive': Permission denied.
 0:15.56 INFO: Setting test_root to /data/local/tmp/test_root
 0:38.84 INFO: Setting run_as_package to org.mozilla.fennec_aurora
 0:40.56 INFO: Resetting test root from /data/local/tmp/test_root to /data/data/org.mozilla.fennec_aurora/test_root
 0:40.66 ERROR: Unable to install /home/upsuper/.mozilla/mozregression/persist/2018-09-05--mozilla-central--fennec-64.0a1.multi.android-arm.apk (error: Unable to set test_root to /data/data/org.mozilla.fennec_aurora/test_root)
 0:40.66 ERROR: Unable to install /home/upsuper/.mozilla/mozregression/persist/2018-09-05--mozilla-central--fennec-64.0a1.multi.android-arm.apk (error: Unable to set test_root to /data/data/org.mozilla.fennec_aurora/test_root)

Alright! Thanks! Can you please attach the output from

adb logcat -c
mozregression
adb logcat -d > logcat.txt

adb 1.0.39 is pretty old but let's not change it at the moment.

I was able to reproduce the error over the weekend using pip install --user mozregression but could not using my original branch from my github repo. I checked if the current master and my older branch still matched and they did apart from the changes wlach made to pin the requirements. I uninstalled mozregression and the other moz* packages along with taskcluster then installed mozregression again. Apart from taskcluster issues with taskId errors I was able to use mozregression with --app gve.

xidorn: Can you check your versions of the following?

mozfile >=2.0.0 but 2.1.0 available
mozinfo >= 1.1.0 but 1.2.2 available
mozlog >= 4.0 but 6.1 available
mozprofile >= 2.2.0 but 2.5.0 available
mozversion >= 2.1.0 but 2.3.0 available
taskcluster >= 6.0.0 but 36.0.0 available

If you could check which version of each package you have installed and then update each to the latest version and then check if the problem still exists before updating the next to see if any one package in particular is the cause for the problem that would be awesome. Something like

# pseudo-code

for package in mozfile mozinfo mozlog mozprofile mozversion taskcluster; do 
    pip show $package
    pip install --user --upgrade $package
    mozregression --app gve --good 2020-08-01 --bad 2020-08-03 --build-type debug
done

Or we can just release an updated mozregression with all of the minimums updated. wlach is out today so we should follow up with him on Tuesday.

Xidorn: I tried to set up the environment with the older packages in order to reproduce the failure but haven't been able to do so. I think if you do

pip install --user --force-reinstall --upgrade mozregression

might fix things for you. Please try it out and let us know so we can release a new version with updated dependencies if we need to do so.

Unfortunately, after upgrading all the dependencies, it still seems to have the same issue. I've upgraded every single package you mentioned in comment 16, and then reinstalled it with the command in comment 17 (except that I'm using pip3 rather than pip), and still no luck.

I presume you are on Ubuntu of some version. I'm running Fedora so the default Python is Python 3 and pip is actually pip3 for me.

More information about your system and device might help.

The output of pip list would show the installed packages on you system.

Information about the device such as the output from ls -la /data/local/tmp and the output of adb shell getprop might be helpful.

Running mozregression with a --debug argument and capturing adb logcat -d * might help as well.

I would definitely upgrade android platform-tools to the latest, uninstall all of the mozilla packages, then install mozregression and see if that helps. wlach suggested that mach mozregression from a source checkout might help eliminate dependency issues.

I don't know what else to say. I think you will have to figure out the problem on your own or contact me directly so we can work this out together.
I am available on chat.mozilla.org and would be happy to help you in real time. My nickname is bc. Either open a private conversation or contact me in CI Automation or send an email directly to my bugmail address.

I downloaded the apk mentioned in comment 13. I found that it is not debuggable...

$ adb shell
walleye:/ $ pm list packages org.mozilla
package:org.mozilla.fennec_aurora
package:org.mozilla.fenix
package:org.mozilla.geckoview_example
package:org.mozilla.geckoview.test
walleye:/ $ am set-debug-app org.mozilla.fennec_aurora
walleye:/ $ echo $?
0
walleye:/ $ run-as org.mozilla.fennec_aurora
run-as: package not debuggable: org.mozilla.fennec_aurora

The causes the problem attempting to set the test_root since we by default will attempt to use run-as on unrooted devices which requires the app to be debuggable.

It is possible we might be able to work around the lack of debuggability by falling back to either /data/local/tmp/test_root or /sdcard/test_root for the test_root. I question the value of worrying about such old apps however.

wlach: I'll let you make the call on if mozdevice should be modified to support this use-case. If so, please move this but to mozbase and I'll take it.

Yeah, this seems like a mozbase issue. I say we can leave it open, but I would classify as a P4 (i.e. "would accept a patch if reasonable") unless/until a non-fennec use case comes up.