Closed
Bug 1383266
Opened 7 years ago
Closed 7 years ago
decom a couple scl3 windows tests masters
Categories
(Infrastructure & Operations Graveyard :: CIDuty, task, P4)
Infrastructure & Operations Graveyard
CIDuty
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: kmoir, Unassigned)
References
Details
Attachments
(3 files)
(deleted),
patch
|
spacurar
:
review+
aobreja
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
spacurar
:
review+
mtabara
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
spacurar
:
review+
aobreja
:
checked-in+
|
Details | Diff | Splinter Review |
We should evaluate load, but may be able to decomm some scl3 windows test masters after we have migrated some win32/win64 tests to tc as tier1 on trunk next week.
Here is the current list
buildbot-master69.bb.releng.use1.mozilla.com:/builds/buildbot/tests1-windows/master
buildbot-master109.bb.releng.scl3.mozilla.com:/builds/buildbot/tests1-windows/master
buildbot-master110.bb.releng.scl3.mozilla.com:/builds/buildbot/tests1-windows/master
buildbot-master111.bb.releng.scl3.mozilla.com:/builds/buildbot/tests1-windows/master
buildbot-master112.bb.releng.scl3.mozilla.com:/builds/buildbot/tests1-windows/master
buildbot-master119.bb.releng.scl3.mozilla.com:/builds/buildbot/tests1-windows/master
buildbot-master126.bb.releng.scl3.mozilla.com:/builds/buildbot/tests1-windows/master
buildbot-master127.bb.releng.scl3.mozilla.com:/builds/buildbot/tests1-windows/master
Comment 1•7 years ago
|
||
Will hold on to grab this bug until the EOW when we can evaluate how the TC migration went. That's due on Wednesdays.
Comment 2•7 years ago
|
||
So we have for buildbot windows:
-150 t-w732-ix
-345 t-w864-ix
-40 t-xp32-ix (and only first 10 are enabled)
From the list above mentiond by Kim we use buildbot-master69 for test purpose,anyway this master was not used in production.
We have for the moment 7 bm masters,we can reduce this number to 3 or less,maybe keeping only :
buildbot-master109.bb.releng.scl3.mozilla.com:/builds/buildbot/tests1-windows/master
buildbot-master110.bb.releng.scl3.mozilla.com:/builds/buildbot/tests1-windows/master
buildbot-master111.bb.releng.scl3.mozilla.com:/builds/buildbot/tests1-windows/master
Flags: needinfo?(kmoir)
Comment 3•7 years ago
|
||
We also have t-1064-ix in scl3, so we should make sure we have enough capacity for those.
Comment 4•7 years ago
|
||
yes sorry also 75 t-1064-ix machines,I forgot to mention them,thank you Amy
Reporter | ||
Comment 5•7 years ago
|
||
That would probably be fine, if you want to be super cautious you could remove two, re-evaluate load and then see if you can remove another. As a side note, Joel wants to enable some more buildbot tests in bug 1393198
Flags: needinfo?(kmoir)
Comment 6•7 years ago
|
||
Will keep for the moment bm109 ,bm110 and bm111.
mysql> select masters.nickname, count(*), masters.enabled from slaves, masters where masters.nickname like '%windows%' and slaves.current_masterid = masters.masterid group by masters.nickname;
+----------------------+----------+---------+
| nickname | count(*) | enabled |
+----------------------+----------+---------+
| bm109-tests1-windows | 86 | 1 |
| bm110-tests1-windows | 84 | 1 |
| bm111-tests1-windows | 82 | 1 |
| bm112-tests1-windows | 81 | 1 |
| bm119-tests1-windows | 81 | 1 |
| bm126-tests1-windows | 81 | 1 |
| bm127-tests1-windows | 80 | 1 |
Comment 7•7 years ago
|
||
Puppet patch to remove bm112,bm119,bm126,bm127
Attachment #8901136 -
Flags: review?(spacurar)
Updated•7 years ago
|
Attachment #8901136 -
Flags: review?(spacurar) → review+
Updated•7 years ago
|
Attachment #8901153 -
Flags: review?(spacurar) → review+
Comment 9•7 years ago
|
||
Disable nagios alerts for BB masters
Attachment #8901161 -
Flags: review?(spacurar)
Updated•7 years ago
|
Attachment #8901161 -
Flags: review?(spacurar) → review+
Comment 10•7 years ago
|
||
Disabled yesterday these 4 bm masters and the masters left seems to handle the machines left on window:
+----------------------+----------+---------+
| nickname | count(*) | enabled |
+----------------------+----------+---------+
| bm109-tests1-windows | 191 | 1 |
| bm110-tests1-windows | 189 | 1 |
| bm111-tests1-windows | 187 | 1 |
| bm128-tests1-windows | 134 | 1 |
| bm129-tests1-windows | 134 | 1 |
| bm137-tests1-windows | 133 | 1 |
| bm138-tests1-windows | 133 | 1 |
| bm139-tests1-windows | 133 | 1 |
| bm140-tests1-windows | 133 | 1 |
+----------------------+----------+---------+
We had a spike of 2k pending jobs on w732 but only for 15 minutes so I will continue with the decom.
Updated•7 years ago
|
Attachment #8901136 -
Flags: checked-in+
Updated•7 years ago
|
Attachment #8901161 -
Flags: checked-in+
Comment 11•7 years ago
|
||
Also deleted from slavealloc.
Comment 12•7 years ago
|
||
Comment on attachment 8901153 [details] [diff] [review]
bug1383266_tools.patch
https://hg.mozilla.org/build/tools/rev/506b046da7bb
Attachment #8901153 -
Flags: checked-in+
Comment 13•7 years ago
|
||
Note explaining the priority level: P4 doesn't mean we've lowered the priority, but the contrary. However, we're aligning these levels to the buildduty quarterly deliverables, where P1-P3 are taken by our daily waterline KTLO operational tasks.
Priority: -- → P4
Comment 14•7 years ago
|
||
All done here.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Updated•7 years ago
|
Product: Release Engineering → Infrastructure & Operations
Updated•5 years ago
|
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•