I am experimenting with the android emulator on packet.net. I'll use this bug to capture some notes.

After signing on to packet.net with Joel's credentials (mine were delayed), I deployed a type 0 server running Ubuntu 16.04 following https://help.packet.net/quick-start/deploy-a-server I tried "kvm-ok" first -- command not found. apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils -- failed apt-get update -- okay apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils -- okay Now: root@gbrownubuntu1604t0:~# kvm-ok INFO: /dev/kvm exists KVM acceleration can be used Great! I downloaded our emulator and avd from tooltool: wget https://raw.githubusercontent.com/mozilla/build-tooltool/master/tooltool.py python tooltool.py fetch --manifest emulator.manifest --authentication-file ttauth python tooltool.py fetch --manifest avd.manifest export PATH=~/android-sdk-linux/emulator:~/android-sdk-linux/tools:~/android-sdk-linux/platform-tools:$PATH and edited test-1.ini paths. Emulator wouldn't start -- missing libGL.so -- so apt-get install libgl1-mesa-glx Now: root@gbrownubuntu1604t0:~# emulator -list-avds test-1 root@gbrownubuntu1604t0:~# emulator -accel-check accel: 0 KVM (version 12) is installed and usable. accel Great!! But: root@gbrownubuntu1604t0:~# emulator -avd test-1 -skip-adb-auth -verbose -show-kernel segfaulted, complaining about the gpu. root@gbrownubuntu1604t0:~# emulator -avd test-1 -skip-adb-auth -verbose -show-kernel -gpu swiftshader got much further but still did not start, complaining QXcbConnection: Could not connect to display apt-get install xvfb xvfb-run emulator ... > emulator.log & root@gbrownubuntu1604t0:~# adb devices List of devices attached emulator-5554 device root@gbrownubuntu1604t0:~# adb shell generic_x86_64:/ $

apt-get install python-pip pip install virtualenv virtualenv venv cd venv/bin . activate pip install mozterm pip install six

python tooltool.py fetch --manifest hostutils.manifest mkdir hostutils cd hostutils tar xvf ../host-utils-60.0a2.en-US.linux-x86_64.tar.gz

My manifests: (venv) root@gbrownubuntu1604t0:~# cat hostutils.manifest [ { "size": 74983178, "visibility": "public", "digest": "800acc77951dbd47880ded6ebfb70b2b6a1e034a52cb0e84fda1931e1825152853c0b5cfa4d05afdc2038f30c1f077fd037d9a49751505d98cbeb2b7e0377328", "algorithm": "sha512", "filename": "host-utils-60.0a2.en-US.linux-x86_64.tar.gz", "unpack": true } ] (venv) root@gbrownubuntu1604t0:~# cat avd.manifest [ { "size": 445624230, "visibility": "public", "digest": "fb951185e21ba6124d8c8ba42202613312401c65e56d52480cc08c87816936b9dd00024f4fc00183ee0e199c91ed5b9209867983c1d13daf5b34bcdc14ee9fb6", "algorithm": "sha512", "filename": "AVDs-x86-android-7.0-build-2017-12-08.tar.gz", "unpack": true } ] (venv) root@gbrownubuntu1604t0:~# cat emulator.manifest [ { "size": 237849484, "visibility": "internal", "digest": "596db3063758aea49f441ed3dccc5429408e037ba953ab8ea676969c12ed267416a7b1f7d61ad5a06fd2c0fe234131843bd9c57a33b8dd8677aa0b25bbb799d7", "algorithm": "sha512", "filename": "android-sdk_r26b-linux.tar.gz", "unpack": "True" } ] Note that emulator has internal visibility -- use an appropriate auth token.

Downloaded Android x86 target.apk and test zips from mozilla-central. adb install -r target.apk mkdir tests unzip the test zips in tests first run of runtestsremote.py failed: xpcshell crashed, missing libgtk.so apt-get install libgtk-3-0 python -u ~/tests/mochitest/runtestsremote.py --app=org.mozilla.fennec_aurora --remote-webserver=10.0.2.2 --xre-path=~/hostutils/host-utils-60.0a2.en-US.linux-x86_64 --utility-path=~/hostutils/host-utils-60.0a2.en-US.linux-x86_64 --certificate-path=~/tests/certs --symbols-path=https://queue.taskcluster.net/v1/task/Gb94v26tTWqfrr18Jni3Tg/artifacts/public/build/target.crashreporter-symbols.zip --quiet --extra-profile-file=fonts --extra-profile-file=hyphenation --screenshot-on-fail --flavor=chrome ... TEST-SKIP | xpfe/appshell/test/test_windowlessBrowser.xul | took 0ms Running manifest: devtools/shared/security/tests/chrome/chrome.ini pushing /root/tests/mochitest/ to /mnt/sdcard/tests/chrome on device... runtests.py | Failed to copy /root/fonts to profile runtests.py | Failed to copy /root/hyphenation to profile pk12util: PKCS12 IMPORT SUCCESSFUL MochitestServer : launching [u'/root/hostutils/host-utils-60.0a2.en-US.linux-x86_64/xpcshell', '-g', '/root/hostutils/host-utils-60.0a2.en-US.linux-x86_64', '-f', '/root/hostutils/host-utils-60.0a2.en-US.linux-x86_64/components/httpd.js', '-e', "const _PROFILE_PATH = '/tmp/tmpE9FWm2.mozrunner'; const _SERVER_PORT = '8888'; const _SERVER_ADDR = '10.0.2.2'; const _TEST_PREFIX = undefined; const _DISPLAY_RESULTS = false;", '-f', '/root/tests/mochitest/server.js'] runtests.py | Server pid: 16965 runtests.py | Websocket server pid: 16968 runtests.py | SSL tunnel pid: 16971 runtests.py | Running with e10s: False runtests.py | Running tests: start. INFO | automation.py | Application pid: 2310 INFO | automation.py | Application ran for: 0:00:17.730436 INFO | zombiecheck | Reading PID log: /tmp/tmpbavRgbpidlog MOZ_UPLOAD_DIR not defined; tombstone check skipped Automation Error: No crash directory (/mnt/sdcard/tests/profile/minidumps) found on remote device Stopping web server Stopping web socket server Stopping ssltunnel leakcheck | refcount logging is off, so leaks can't be detected! runtests.py | Running tests: end. Unable to retrieve log file (/mnt/sdcard/tests/logs/mochitest.log) from remote device Buffered messages finished Running manifest: dom/animation/test/chrome.ini runtests.py | Failed to copy /root/fonts to profile runtests.py | Failed to copy /root/hyphenation to profile pk12util: PKCS12 IMPORT SUCCESSFUL MochitestServer : launching [u'/root/hostutils/host-utils-60.0a2.en-US.linux-x86_64/xpcshell', '-g', '/root/hostutils/host-utils-60.0a2.en-US.linux-x86_64', '-f', '/root/hostutils/host-utils-60.0a2.en-US.linux-x86_64/components/httpd.js', '-e', "const _PROFILE_PATH = '/tmp/tmpaZaP4u.mozrunner'; const _SERVER_PORT = '8888'; const _SERVER_ADDR = '10.0.2.2'; const _TEST_PREFIX = undefined; const _DISPLAY_RESULTS = false;", '-f', '/root/tests/mochitest/server.js'] runtests.py | Server pid: 17109 runtests.py | Websocket server pid: 17112 runtests.py | SSL tunnel pid: 17115 runtests.py | Running with e10s: False runtests.py | Running tests: start. INFO | automation.py | Application pid: 0 INFO | automation.py | Application ran for: 0:00:14.435885 INFO | zombiecheck | Reading PID log: /tmp/tmpI9qtq_pidlog MOZ_UPLOAD_DIR not defined; tombstone check skipped PROCESS-CRASH | remoteautomation.py | java-exception Could not open database at android.database.sqlite.SQLiteConnection.nativeOpen(Native Method) Stopping web server Stopping web socket server Stopping ssltunnel leakcheck | refcount logging is off, so leaks can't be detected! runtests.py | Running tests: end.

(In reply to Geoff Brown [:gbrown] from comment #5) I got stuck here. Firefox was crashing on startup and my fanciful imagination led me to investigate various unlikely scenarios before I remembered: runtime permissions! > Downloaded Android x86 target.apk and test zips from mozilla-central. > > adb install -r target.apk adb install -r -g target.apk !!

Now I can start an Android 7.0 x86 emulator on a type 0 or type 1 instance and run various tests, semi-manually (I use custom scripts to run the emulator and run the test harnesses, based on manual download/install of a build + test zips: I'm not using docker and I'm not using mozharness.) Providing meaningful timing benchmarks is tricky because, like any new platform, some tests fail, some crash, some hang. Thankfully, jsreftests generally pass. Recall that in our current continuous integration, our Android/opt armv7 builds run jsreftests in 40 chunks with a total run time of about 1800 minutes (30 hours!). From https://bugzilla.mozilla.org/show_bug.cgi?id=1438665#c4, recall that an Android/opt x86 build ran jsreftests on the Talos moonshot hardware in about 100 minutes (18x faster than current armv7). On a packet.net type 0 instance, I used an Android/opt x86 build to run jsreftests in about 360 minutes (5x faster than current armv7). On a packet.net type 1 instance, I used an Android/opt x86 build to run jsreftests in about 80 minutes (22x faster than current armv7).

Similarly, crashtests pass consistently on all platforms. Current Android/opt armv7 crashtests: about 120 minutes Talos moonshot x86: 7 minutes packet.net type 0 x86: 30 minutes packet.net type 1 x86: 6 minutes

mochitest-chrome has some failures on the new platforms, so maybe it's not a good measure, but... Current Android/opt armv7 mochitest-chrome: about 100 minutes Talos moonshot x86: 20 minutes packet.net type 0 x86: 26 minutes packet.net type 1 x86: 25 minutes

There are lots of timeouts on plain mochitests on the new platforms, so I can't compare them as a whole. However, selecting one large-ish manifest that passes everywhere - dom-level2-html mochitests - I see these numbers: Current Android/opt armv7: 80 minutes Talos moonshot x86: 2 minutes packet.net type 0 x86: 14 minutes packet.net type 1 x86: 2 minutes

In summary, these experiments were very promising: - it is possible to run an Android 7.0 x86 emulator with kvm acceleration on packet.net type 0 or type 1 servers; - performance varies by test type, but type 1 results were similar to moonshot performance (bug 1438665); type 0 results were sometimes about 5x slower, but still considerably faster than our current CI arm emulators. :coop's team is now trying to provide taskcluster/try access to packet.net (hoping for news in Q2).