Closed Bug 1639955 Opened 5 years ago Closed 5 years ago

[fenix] Perma browsertime failure Exception: Download failed, no more retries!

Categories

(Testing :: Raptor, defect, P1)

defect

Tracking

(firefox-esr68 unaffected, firefox76 unaffected, firefox77 unaffected, firefox78 fixed)

RESOLVED FIXED
mozilla78
Tracking Status
firefox-esr68 --- unaffected
firefox76 --- unaffected
firefox77 --- unaffected
firefox78 --- fixed

People

(Reporter: sparky, Assigned: sparky)

References

(Regression)

Details

(Keywords: regression)

Attachments

(2 files)

Failure in Fenix, the geckodriver path changed again...

Removing /builds/task_1590080588/fetches/browsertime.tar.bz2
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OGsfn5JTTmuPnkAoITV6Qw/artifacts/public/ffmpeg-4.1.4-i686-static.tar.xz resolved to 16633296 bytes with sha256 f7a2f0f8f91506d67f80fbe5b8a5c264d0fb60cb3cd258d03344890fbd55e6fd in 4.481s
Extracting /builds/task_1590080588/fetches/ffmpeg-4.1.4-i686-static.tar.xz to /builds/task_1590080588/fetches
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/d8NbOplyTQubsa-WdF8ugg/artifacts/public/build/node.tar.xz resolved to 14185872 bytes with sha256 649362969c6bc729967e5ca1570432a20932be6792e99f5b5559267baefa2295 in 3.248s
Extracting /builds/task_1590080588/fetches/node.tar.xz to /builds/task_1590080588/fetches
/builds/task_1590080588/fetches/ffmpeg-4.1.4-i686-static.tar.xz extracted in 1.371s
Removing /builds/task_1590080588/fetches/ffmpeg-4.1.4-i686-static.tar.xz
/builds/task_1590080588/fetches/node.tar.xz extracted in 1.128s
Removing /builds/task_1590080588/fetches/node.tar.xz
attempt 2/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Utfw7DA9SVGCeeqfUM6rRQ/artifacts/public/build/geckodriver.tar.gz to /builds/task_1590080588/fetches/geckodriver.tar.gz
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Utfw7DA9SVGCeeqfUM6rRQ/artifacts/public/build/geckodriver.tar.gz
Download failed: HTTP Error 404: Not Found
sleeping for 90.00s (attempt 2/5)
attempt 3/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Utfw7DA9SVGCeeqfUM6rRQ/artifacts/public/build/geckodriver.tar.gz to /builds/task_1590080588/fetches/geckodriver.tar.gz
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Utfw7DA9SVGCeeqfUM6rRQ/artifacts/public/build/geckodriver.tar.gz
Download failed: HTTP Error 404: Not Found
sleeping for 136.00s (attempt 3/5)
attempt 4/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Utfw7DA9SVGCeeqfUM6rRQ/artifacts/public/build/geckodriver.tar.gz to /builds/task_1590080588/fetches/geckodriver.tar.gz
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Utfw7DA9SVGCeeqfUM6rRQ/artifacts/public/build/geckodriver.tar.gz
Download failed: HTTP Error 404: Not Found
sleeping for 201.50s (attempt 4/5)
attempt 5/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Utfw7DA9SVGCeeqfUM6rRQ/artifacts/public/build/geckodriver.tar.gz to /builds/task_1590080588/fetches/geckodriver.tar.gz
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Utfw7DA9SVGCeeqfUM6rRQ/artifacts/public/build/geckodriver.tar.gz
Download failed: HTTP Error 404: Not Found
Traceback (most recent call last):
  File "./fetch-content", line 675, in <module>
    sys.exit(main())
  File "./fetch-content", line 671, in main
    return args.func(args)
  File "./fetch-content", line 605, in command_task_artifacts
    fetch_urls(downloads)
  File "./fetch-content", line 477, in fetch_urls
    f.result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "./fetch-content", line 456, in fetch_and_extract
    download_to_path(url, dest_path, sha256=sha256, size=size)
  File "./fetch-content", line 236, in download_to_path
    raise Exception("Download failed, no more retries!")
Exception: Download failed, no more retries!
[fetches 2020-05-21T18:22:17.750Z] removing /builds/task_1590080588/fetches
[fetches 2020-05-21T18:22:17.968Z] finished
Traceback (most recent call last):
  File "./run-task", line 1006, in <module>
    sys.exit(main(sys.argv[1:]))
  File "./run-task", line 994, in main
    fetch_artifacts()
  File "./run-task", line 653, in fetch_artifacts
    subprocess.run(cmd, check=True, env=os.environ)
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/bin/python3', '-u', './fetch-content', 'task-artifacts']' returned non-zero exit status 1.
Flags: needinfo?(mozilla)

I had another look and it's indeed the change from public/build to public/. Until now I never got a reply from Tom, why that was necessary. Let me ask that on the other bug. And I think that this patch was wrong, because builds have to exist under public/build.

Ah ok thanks, if this could be fixed on the mozilla-central side that would be ideal

We originally switched it to use just public/ becuase iscript broke with the
extra path, but that broke out-of-tree consumers. Now that iscript is fixed,
switch it back.

Pushed by mozilla@hocat.ca: https://hg.mozilla.org/integration/autoland/rev/8b12cbd16f06 Include `toolchain-artifact` in the cached-task digest; r=aki https://hg.mozilla.org/integration/autoland/rev/cffc2f23fdbd Use prefix for geckodriver toolchain artifacts; r=aki
Flags: needinfo?(mozilla)
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla78

Set release status flags based on info from the regressing bug 1577110

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: