I'm currently trying to deploy gecko-t-win10-64-gpu-b instances in OpenCloudConfig. I consistently get a timeout waiting for AMI to build. I expect the currently running task to hit the same issue - see https://tools.taskcluster.net/groups/ZvnZQbXaRcKZwFS3598pvA/tasks/IOrW91XcSI-OOmdkUZKz1g/runs/0/logs/public%2Flogs%2Flive.log The only logging I see from the instance creation in papertrail is: Dec 04 16:57:00 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: generic-worker is not running. Dec 04 16:57:00 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: OpenCloudConfig is running. Dec 04 16:57:00 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: instance appears to be initialising. Dec 04 17:03:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: generic-worker is not running. Dec 04 17:03:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: OpenCloudConfig is running. Dec 04 17:03:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: instance appears to be initialising. Dec 04 17:05:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: generic-worker is not running. Dec 04 17:05:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: OpenCloudConfig is running. Dec 04 17:05:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: instance appears to be initialising. Dec 04 17:07:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: generic-worker is not running. Dec 04 17:07:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: OpenCloudConfig is running. Dec 04 17:07:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: instance appears to be initialising. Dec 04 17:13:00 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: generic-worker is not running. Dec 04 17:13:00 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: OpenCloudConfig is running. Dec 04 17:13:00 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: instance appears to be initialising. Dec 04 17:19:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: generic-worker is not running. Dec 04 17:19:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: OpenCloudConfig is running. Dec 04 17:19:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: instance appears to be initialising. Dec 04 17:25:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: generic-worker is not running. Dec 04 17:25:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: OpenCloudConfig is running. Dec 04 17:25:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: instance appears to be initialising. Dec 04 17:29:00 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: generic-worker is not running. Dec 04 17:29:00 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: OpenCloudConfig is running. Dec 04 17:29:00 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: instance appears to be initialising. Dec 04 17:31:00 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: generic-worker is not running. Dec 04 17:31:00 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: OpenCloudConfig is running. Dec 04 17:31:00 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: instance appears to be initialising. Dec 04 17:49:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: generic-worker is not running. Dec 04 17:49:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: Is-ConditionTrue :: OpenCloudConfig is running. Dec 04 17:49:01 i-0d33f89708f6b73f3.gecko-t-win10-64-gpu-b.usw2.mozilla.com HaltOnIdle: instance appears to be initialising. In order to troubleshoot, I'd like to log onto the machine. However, I am not able to log on with the Administrator account that I have for the win10 base instance (password from October 10), and since OCC seems not to have run, I don't have a new password for the golden instance. https://tools.taskcluster.net/secrets/repo%3Agithub.com%2Fmozilla-releng%2FOpenCloudConfig%3Agecko-t-win10-64-gpu-b is also not updated. So the problem is two-fold: 1) something appears to be stalled/not working 2) I don't have working credentials to log onto the machine to investigate If you're able to help me with either of the above, that would be great. Many thanks!

OCC deploy log: https://tools.taskcluster.net/groups/ZvnZQbXaRcKZwFS3598pvA/tasks/IOrW91XcSI-OOmdkUZKz1g/runs/0/logs/public%2Flogs%2Flive.log Instance: https://us-west-2.console.aws.amazon.com/ec2/v2/home?region=us-west-2#Instances:instanceId=i-0d33f89708f6b73f3;sort=instanceId Papertrail log: https://papertrailapp.com/groups/2488493/events?q=i-0d33f89708f6b73f3 Secret: https://tools.taskcluster.net/secrets/repo%3Agithub.com%2Fmozilla-releng%2FOpenCloudConfig%3Agecko-t-win10-64-gpu-b IP: 34.210.26.91

Rob (:grenade) highlighted where I can get the password from, and I was able to retrieve it, and troubleshoot the issue. Thanks Rob!