Closed Bug 1437071 Opened 7 years ago Closed 5 years ago

Move all workers to using the blob storageType artifact system

Categories

(Taskcluster :: Workers, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: jhford, Unassigned, Mentored)

References

Details

We have the new artifact api and it has. We should move all workers over to this. The remaining work is: 1. Deprecate S3/Azure artifacts https://github.com/taskcluster/taskcluster-queue/issues/248 2. Provide client libraries enough information to do safe downloads: https://github.com/taskcluster/taskcluster-queue/pull/233 3. Once 2. is complete: add arbitrary artifact type download support to JS (https://github.com/taskcluster/taskcluster-lib-artifact/issues/4) and Go (https://github.com/taskcluster/taskcluster-lib-artifact-go/issues/2) 4. Move docker-worker to use taskcluster-lib-artifact once 3. is completed 5. Move generic-worker to use taskcluster-lib-artifact-go once 3. is completed Wander, I remember you were working on 4. a little while back, do you have a bug, PR or issue for that? Pete, do you know of a bug, PR or issue for tracking this work in generic-worker
Flags: needinfo?(wcosta)
Flags: needinfo?(pmoore)
I have a patch for it https://github.com/walac/docker-worker/commit/12afb6c55d435e9e5952f1b446efd9280fae9d8d But iirc, tests were failing due to some permission issue I couldn't figure out.
Flags: needinfo?(wcosta)
(In reply to John Ford [:jhford] CET/CEST Berlin Time from comment #0) > Pete, do you know of a bug, PR or issue for tracking this work in > generic-worker I've created bug 1437461 for this. Thanks!
Depends on: 1437461
Flags: needinfo?(pmoore)
John: where are we at with this? Any update?
Flags: needinfo?(jhford)
Now that we have server side support for artifact type determination, we can update lib-artifact[-go] to support non-blob artifacts. Once that's happened we can integrate that library into workers and then we should be done here.
Flags: needinfo?(jhford)
I've added support for non-blob artifacts today in https://github.com/taskcluster/taskcluster-lib-artifact-go/pull/8 . When that merges, there nothing block go clients from moving forward with implementing blob-artifacts.

This will be necessary before we start using the Object Service, but that's not for a few months yet. Fixing this in the generic-worker (which is in Golang) would be a good mentored task during those few months.

Mentor: dustin
Component: General → Workers
Blocks: 1585578

We're going to stick to S3 artifacts until we build the Object service.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.