There are some failures caused by some geckodriver artifact changes that haven't yet been fixed in the fenix repo: https://treeherder.mozilla.org/#/jobs?repo=fenix&tier=1%2C2%2C3&searchStr=btime&revision=f63cb4e09f84e09eed0c8bfccc01cf1ebf6a62ed

Extracting /builds/task_1588895281/fetches/browsertime.tar.bz2 to /builds/task_1588895281/fetches
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/BwoWT4lLRyu34vdlwsJe8Q/artifacts/public/ffmpeg-4.1.4-i686-static.tar.xz resolved to 16633296 bytes with sha256 f7a2f0f8f91506d67f80fbe5b8a5c264d0fb60cb3cd258d03344890fbd55e6fd in 5.756s
Extracting /builds/task_1588895281/fetches/ffmpeg-4.1.4-i686-static.tar.xz to /builds/task_1588895281/fetches
/builds/task_1588895281/fetches/node.tar.xz extracted in 1.822s
Removing /builds/task_1588895281/fetches/node.tar.xz
/builds/task_1588895281/fetches/ffmpeg-4.1.4-i686-static.tar.xz extracted in 1.527s
Removing /builds/task_1588895281/fetches/ffmpeg-4.1.4-i686-static.tar.xz
/builds/task_1588895281/fetches/browsertime.tar.bz2 extracted in 2.309s
Removing /builds/task_1588895281/fetches/browsertime.tar.bz2
attempt 2/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/KjXSn7nYTr2apHk6ax2q6A/artifacts/public/build/geckodriver.tar.xz to /builds/task_1588895281/fetches/geckodriver.tar.xz
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/KjXSn7nYTr2apHk6ax2q6A/artifacts/public/build/geckodriver.tar.xz
Download failed: HTTP Error 404: Not Found
sleeping for 90.00s (attempt 2/5)
attempt 3/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/KjXSn7nYTr2apHk6ax2q6A/artifacts/public/build/geckodriver.tar.xz to /builds/task_1588895281/fetches/geckodriver.tar.xz
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/KjXSn7nYTr2apHk6ax2q6A/artifacts/public/build/geckodriver.tar.xz
Download failed: HTTP Error 404: Not Found
sleeping for 135.00s (attempt 3/5)
attempt 4/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/KjXSn7nYTr2apHk6ax2q6A/artifacts/public/build/geckodriver.tar.xz to /builds/task_1588895281/fetches/geckodriver.tar.xz
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/KjXSn7nYTr2apHk6ax2q6A/artifacts/public/build/geckodriver.tar.xz
Download failed: HTTP Error 404: Not Found
sleeping for 203.50s (attempt 4/5)
attempt 5/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/KjXSn7nYTr2apHk6ax2q6A/artifacts/public/build/geckodriver.tar.xz to /builds/task_1588895281/fetches/geckodriver.tar.xz
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/KjXSn7nYTr2apHk6ax2q6A/artifacts/public/build/geckodriver.tar.xz
Download failed: HTTP Error 404: Not Found
Traceback (most recent call last):
  File "./fetch-content", line 675, in <module>
    sys.exit(main())
  File "./fetch-content", line 671, in main
    return args.func(args)
  File "./fetch-content", line 605, in command_task_artifacts
    fetch_urls(downloads)
  File "./fetch-content", line 477, in fetch_urls
    f.result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "./fetch-content", line 456, in fetch_and_extract
    download_to_path(url, dest_path, sha256=sha256, size=size)
  File "./fetch-content", line 236, in download_to_path
    raise Exception("Download failed, no more retries!")
Exception: Download failed, no more retries!
[fetches 2020-05-08T01:20:55.164Z] removing /builds/task_1588895281/fetches
[fetches 2020-05-08T01:20:55.391Z] finished
Traceback (most recent call last):
  File "./run-task", line 1006, in <module>
    sys.exit(main(sys.argv[1:]))
  File "./run-task", line 994, in main
    fetch_artifacts()
  File "./run-task", line 653, in fetch_artifacts
    subprocess.run(cmd, check=True, env=os.environ)
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/bin/python3', '-u', './fetch-content', 'task-artifacts']' returned non-zero exit status 1.

This possibly regressed by bug 1577110 given that we weren't aware of the Fenix repository. Sorry for that.

As a quick fix simply use the tar.gz file extension instead of xz. Note that we also index the signed binaries now:

https://firefox-ci-tc.services.mozilla.com/tasks/index/gecko.v2.mozilla-central.latest.geckodriver

Thanks, I've got a patch up here: https://github.com/mozilla-mobile/fenix/pull/10558

PR landed - it tested browsertime in it and everything's good now.