So very very awesome. Thanks for saving resources! In https://treeherder.mozilla.org/#/jobs?repo=try&revision=055ceb2c6e7f catlee killed a try push, while panda tests were running (nothing special about that cancel, it's just the most recent one which took out a whole bunch). https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=panda-0450 https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=panda-0271 https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=panda-0152 https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=panda-0296 https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=panda-0544 https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=panda-0397 https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=panda-0155 https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=panda-0402 https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=panda-0040 https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=panda-0550 (on and on, I'm getting tired of copy-paste) [[[ 13:08:53 INFO - Got request, url=http://mobile-imaging-006.p6.releng.scl3.mozilla.com/api/request/2959451/ 13:08:53 INFO - Waiting for request 'ready' stage. Current state: 'finding_device' 13:09:53 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:10:53 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:11:54 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:12:54 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:13:54 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:14:54 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:15:54 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:16:54 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:17:55 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:18:55 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:19:55 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:20:55 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:21:55 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:22:55 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:23:55 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:24:55 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:25:55 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:26:55 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:27:55 INFO - Waiting for request 'ready' stage. Current state: 'failed_device_busy' 13:28:56 ERROR - INFRA-ERROR: Request did not become ready in time ]]] for the next five or six (or maybe "however many ran in the next four hours", dunno) jobs. We would be far far better off in capacity terms if we just prohibited cancelling any jobs on pandas, since the longest job takes far less time than the time a cancel takes one out of service.