Manual PDU cycle got this device working again.

17 reds in a row, https://secure.pub.build.mozilla.org/buildapi/recent/tegra-141

ran stop_cp on it, and vin says DCOps will grab it right now, since they are still in the colo

Back in production.

Has a sort of odd failure pattern, according to https://secure.pub.build.mozilla.org/buildapi/recent/tegra-141?numbuilds=50, but it also has a completely unacceptable failure rate.

12 red in row, not counting one killed on try.

14.

16.

ran stop_cp on it

IT handled this, start_cp run now

Fails every run; please re-disable.

Tegra-141 reimaged and pingable + swapped sd card --- tegra-141.build.mtv1.mozilla.com ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss

Its last 217 jobs in a row have all done nothing but set RETRY.

Hi! It's three weeks later, and by not disabling this tegra over the last three weeks, we've allowed it to take 365 jobs, and set RETRY on 365 jobs. Counting the time to take the job and the time for another slave to actually take it and do it, say 7 minutes is a conservative estimate. This slave has slowed us down by 42.5 hours over the last three weeks. Please disable it.

(In reply to Phil Ringnalda (:philor) from comment #14) > Hi! It's three weeks later, and by not disabling this tegra over the last > three weeks, we've allowed it to take 365 jobs, and set RETRY on 365 jobs. > Counting the time to take the job and the time for another slave to actually > take it and do it, say 7 minutes is a conservative estimate. This slave has > slowed us down by 42.5 hours over the last three weeks. > > Please disable it.

disabled

Sending this slave to recovery -->Automated message.

Successfully recovered.

Sunday, October 20, 2013 10:39:07 PM

flashed and reimaged

Back in production

SD card replaced, tegra reimaged and flashed. [vle@admin1a.private.scl3 ~]$ fping tegra-141.tegra.releng.scl3.mozilla.com tegra-141.tegra.releng.scl3.mozilla.com is alive

Power cycled, waited a day. error.flg: Automation Error: Unable to connect to device after 5 attempts Pings but can't connect via telnet.

in production running green it had one retry: Remote Device Error: Unable to copy file '/builds/tegra-141/talos-data/talos/page_load_test/tart/dbghelp.dll' to directory 'dbghelp.dll' on the remote device mozdevice.devicemanager.DMError: Automation Error: Error processing command 'mkdr /dbghelp.dll'; err='Could not create the directory /dbghelp.dll' but this looked like an anomaly

Hasn't taken a job for two weeks.

Disabled in slavealloc to stop the pointless stream of reboots.

SD card formatted, tegra flashed and reimaged. vle@vle-10516 ~ $ telnet tegra-141.tegra.releng.scl3.mozilla.com 20701 Trying 10.26.85.106... Connected to tegra-141.tegra.releng.scl3.mozilla.com. Escape character is '^]'. $>^] telnet> q

Reenabled.