+++ This bug was initially created as a clone of Bug #1317783 +++ We'll have to make these changes: a. scopes For signing, we have 3 levels of permissions, guarded by scopes. project:releng:signing:cert:release-signing [1], which we only allow on release-capable branches, project:releng:signing:cert:nightly-signing [2], which we only allow on nightly-capable branches, and project:releng:signing:cert:dep-signing, which can be used anywhere. Scriptworker uses the above-linked data structures to verify that a privileged scope is only used on an appropriate branch. Signingscript determines which level of access to grant based on those scopes [3]. We need to follow this model for the other *scripts. For pushapk, I think this would be what accounts/products we're allowed to push to. That would allow chain of trust verification to make sure we're not pushing to a privileged account/product from a non-privileged branch. If we can't separate creds at first, let's file a followup bug to do so, and verify our upload bucket location matches release, nightly, or staging based on these scopes. [1] https://github.com/mozilla-releng/scriptworker/blob/121c474f5b21084a4a3742f21c3f30c018e5c766/scriptworker/constants.py#L219 [2] https://github.com/mozilla-releng/scriptworker/blob/121c474f5b21084a4a3742f21c3f30c018e5c766/scriptworker/constants.py#L232 [3] https://github.com/mozilla-releng/signingscript/blob/master/signingscript/task.py#L19 b. downloads / upstreamArtifacts For signing, we used to have task.payload.unsignedArtifacts, which was a list of URLs. Now we have task.payload.upstreamArtifacts [4], which is a list of dictionaries that look like "upstreamArtifacts": [{ "paths": [ "public/build/target.tar.bz2", "public/build/target.checksums" ], "formats": ["gpg"], "taskId": "GFPKeLbAQN2fytGOXgatIg", "taskType": "build" }, { ... }] The taskId is the taskId of the task we're downloading from. The paths are the artifact paths we're downloading. I don't know if you need to use the "formats" key or need to embed any additional information; we can play with this schema. "taskType" is there for chain of trust verification. Currently we only support "build", "l10n", "decision", "docker-image", but we can add more. Scriptworker will pre-download these artifacts into $artifact_dir/public/cot/$task_id/$path , and verify their SHAs before calling the script. Pushapk no longer needs to download these artifacts; it can and should use the pre-downloaded artifacts on disk. I don't know how many artifacts we download, and how large this is going to get. If we don't want to upload all of these upstreamArtifacts at the end of the pushapk task, we can move them to $work_dir or otherwise remove from $artifact_dir before the end of the task, or change where scriptworker downloads them to. [4] https://queue.taskcluster.net/v1/task/M81unWcDQje2XEwhtmDXrw c. scriptworker.cot.verify will need to support pushapk type workers We'll also have to support any other new task types that we depend on. d. upstream tasks will need to point at the right deps and have chain of trust generation enabled. To enable chain of trust generation in a non-scriptworker task, set task.payload.features.ChainOfTrust to true. When there are additional tasks we need to set as chain of trust dependencies in non-scriptworker tasks, we add them to task.extra.ChainOfTrust.inputs, which looks like "inputs": { "docker-image": "taskId", ... } For upstream scriptworker tasks, we have sign_chain_of_trust [5] and upstreamArtifacts. We can also follow the same task.extra.chainOfTrust.inputs model if that's easiest. This may prevent us from fully enabling chain of trust verification on pushapk if we depend directly on other non-signing scriptworker tasks that don't yet have chain of trust enabled, but we have some prefs [6] we can use until it's all enabled end-to-end. [5] https://github.com/mozilla-releng/scriptworker/blob/121c474f5b21084a4a3742f21c3f30c018e5c766/scriptworker/constants.py#L58 [6] https://github.com/mozilla-releng/scriptworker/blob/121c474f5b21084a4a3742f21c3f30c018e5c766/scriptworker/constants.py#L57-L60 e. puppet With bug 1316702, we now have a shared scriptworker puppet module. Let's use that. * There are updated dependencies, all pushed to the python3.5 pypi location. * We now use a scriptworker.yaml which is much larger than our previous config.json. This is populated in the scriptworker module, using variables you pass [7]. I still have the supervisord settings in the signing scriptworker area, because the watch file list can be different per instance type. * gpg keys - we'll need to create new gpg keys per scriptworker instance, and make sure they're signed by an appropriate key. The trusted keys are in scriptworker/trusted and the worker keys go into scriptworker/valid in the cot-gpg-keys repo [8]. [7] https://hg.mozilla.org/build/puppet/file/tip/modules/signing_scriptworker/manifests/init.pp#l54 [8] https://github.com/mozilla-releng/cot-gpg-keys

1 failures in 462 pushes (0.002 failures/push) were associated with this bug in the last 7 days. Repository breakdown: * mozilla-central: 1 Platform breakdown: * Android: 1 For more details, see: https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1428444&startday=2018-01-01&endday=2018-01-07&tree=all