Closed
Bug 683417
Opened 13 years ago
Closed 10 years ago
retry.py didn't actually kill process tree for a timed-out pushsnip
Categories
(Release Engineering :: Release Automation: Other, defect, P2)
Release Engineering
Release Automation: Other
Tracking
(Not tracked)
RESOLVED
WONTFIX
People
(Reporter: nthomas, Assigned: nthomas)
Details
Attachments
(2 files)
(deleted),
text/plain
|
Details | |
(deleted),
patch
|
rail
:
review+
|
Details | Diff | Splinter Review |
In bug 683412 we timed out a pushsnip, but on inspection of the processes list on aus2-staging there was sync to Phoenix still running. This was actually advantageous, but I bet it wasn't expected.
Updated•13 years ago
|
OS: Mac OS X → All
Priority: -- → P3
Hardware: x86 → All
Updated•13 years ago
|
Blocks: hg-automation
Assignee | ||
Comment 1•13 years ago
|
||
Hit this again today. Based on http://superuser.com/questions/20679/why-does-my-remote-process-still-run-after-killing-an-ssh-session, we should give the -t argument to ssh.
Assignee: nobody → nrthomas
Priority: P3 → P2
Assignee | ||
Comment 2•13 years ago
|
||
For 10.0b6 the pushsnip of the test snippets failed on all five of retry attempts, often while pushing to PHX. This should help by not leaving running processes slowing down NFS (ever moar!), and by setting the pushsnip timeout to the same 2 hours that backupsnip gets.
Attachment #591341 -
Flags: review?(rail)
Updated•13 years ago
|
Attachment #591341 -
Flags: review?(rail) → review+
Assignee | ||
Comment 3•13 years ago
|
||
Comment on attachment 591341 [details] [diff] [review]
[buildbotcustom] use -t with ssh, bump timeout for pushsnip
http://hg.mozilla.org/build/buildbotcustom/rev/83f17929c032
Attachment #591341 -
Flags: checked-in+
Comment 4•13 years ago
|
||
This landed in production today.
Assignee | ||
Comment 5•13 years ago
|
||
May not be working, from the 3.6.26 build2 log:
Pseudo-terminal will not be allocated because stdin is not a terminal.
which could be fallout from having
using PTY: False
when making the call from buildbot.
Assignee | ||
Comment 6•13 years ago
|
||
It does look like the processes are being cleaned up properly though. Needs more investigation.
Comment 7•13 years ago
|
||
(In reply to Nick Thomas [:nthomas] from comment #5)
> May not be working, from the 3.6.26 build2 log:
> Pseudo-terminal will not be allocated because stdin is not a terminal.
> which could be fallout from having
> using PTY: False
> when making the call from buildbot.
Likely we need (or want) to add |usePTY=True| to the RetryingShellCommand step specifically.
The default of the steps is of course to use whatever the slave is configured as (and we currently configure it to "do not use a PTY") each ShellCommand-based step can be setup to override this behavior with that arg.
Comment 8•13 years ago
|
||
Mass move of bugs to Release Automation component.
Component: Release Engineering → Release Engineering: Automation (Release Automation)
Flags: checked-in+
Updated•13 years ago
|
No longer blocks: hg-automation
Assignee | ||
Comment 9•12 years ago
|
||
Probably needs to be ssh -tt based on the man page:
-t Force pseudo-tty allocation. This can be used to execute arbitrary screen-based programs on a remote machine, which can be very useful, e.g. when implementing menu services. Multiple -t options force tty allocation, even if ssh has no local tty.
Updated•11 years ago
|
Product: mozilla.org → Release Engineering
Comment 10•10 years ago
|
||
pushsnip is going away in the forseeable future, probably not going to fix this
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → WONTFIX
You need to log in
before you can comment on or make changes to this bug.
Description
•