Closed Bug 1375190 Opened 7 years ago Closed 6 years ago

[docker-worker] Support reading workerType secrets from taskcluster-secrets service

Categories

(Taskcluster :: Workers, enhancement)

Product:
Taskcluster
Component:
Workers
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: pmoore, Assigned: pmoore)

References

Details

Attachments

(1 file)

GitHub Pull Request for docker-worker
(deleted), text/x-github-pull-request
jhford
: review+
Details
See parent bug for more context. Currently it looks like docker-worker reads some (confidential) worker type configuration from the "secrets" portion of its worker type definition. In order to support bug 1375155 in making worker type definitions public information, docker-worker will need to be able to get secrets from the taskcluster-secrets service instead. This assumption is based on finding some secrets in some worker types - if this feature already exists, but not all worker type definitions have been updated to use it, we can close this bug. Rolling out the feature and migrating secrets over will be separate bugs. This bug is purely about implementing this new feature in docker-worker.
Blocks: 1375192
No longer blocks: 1375155
Summary: Support reading workerType secrets from taskcluster-secrets service → [docker-worker] Support reading workerType secrets from taskcluster-secrets service
Found in triage. Pete: you were going to do something with this batch of bugs, IIRC.
Flags: needinfo?(pmoore)
I haven't had bandwidth to work on this, but yes we would certainly like to get to it.
Flags: needinfo?(pmoore)
Component: Docker-Worker → Workers
Attached file GitHub Pull Request for docker-worker (deleted) —

Not ready for review yet, but I've started work on the code changes.

Will flag for review as soon as CI is passing and I'm confident the changes are correct...

Assignee: nobody → pmoore
Status: NEW → ASSIGNED
Comment on attachment 9043587 [details] GitHub Pull Request for docker-worker Thanks guys!
Attachment #9043587 - Flags: review?(wcosta)
Attachment #9043587 - Flags: review?(jhford)
Attachment #9043587 - Flags: review?(jhford) → review+

I've deployed on ami-test and removed the worker type secrets from the worker type definition.

The worker successfully ran task E7ggXxODRvSZuLVQZFzcag, with working live logs, which implies it could successfully retrieve the livelog configuration from the worker type secret in the taskcluster secrets service.

Running a mozilla-central linux64 build for a sanity check:

https://tools.taskcluster.net/groups/ZSLswW7pSwiybCRFtZ0v8g/tasks/ZSLswW7pSwiybCRFtZ0v8g/runs/0/logs/public%2Flogs%2Flive.log

This is just a modified task copied from mozilla-central on treeherder, with the treeherder metadata removed, and the worker type changed to ami-test.

The linux64 job ran successfully, and :wcosta merged the PR.

Still need to make a docker-worker release.

Roll out to all docker-worker worker types is in bug 1375192.

Attachment #9043587 - Flags: review?(wcosta)
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED

For the record, with Pete's help, I created/updated all secrets matching: worker-type:aws-provisioner-v1/mobile-*[1]. This was causing workers to not spin up. I also cleaned the secrets section in the worker definition, by leaving an empty dictionary[2].

[1] For instance: https://tools.taskcluster.net/secrets/worker-type%3Aaws-provisioner-v1%2Fmobile-3-b-ref-browser
[2] For instance: https://tools.taskcluster.net/aws-provisioner/mobile-3-b-ref-browser/view

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: