Closed Bug 979954 Opened 11 years ago Closed 11 years ago

please adjust file age check on buildbot-master65:/builds/slaverebooter/slaverebooter.log

Categories

(mozilla.org Graveyard :: Server Operations, task)

x86_64
Linux
task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: bhearsum, Assigned: ashish)

References

Details

This check is currently alerting if the file is 4h old, but we've discovered that it can be validly older than that. Can we adjust the check to wait until the file is 6h old before alerting?
Blocks: re-nagios
(In reply to Ben Hearsum [:bhearsum] from comment #0) > This check is currently alerting if the file is 4h old, but we've discovered > that it can be validly older than that. Can we adjust the check to wait > until the file is 6h old before alerting? Ben, this check is set for 2 thresholds. Warning = 4hrs, Critical = 6hrs. Sounds like you only want an alert when intervention is needed. Instead of the 2 thresholds, should we just alert when the file_age breaks the 6hr mark period?
(In reply to Rick Bryce [:rbryce] from comment #1) > (In reply to Ben Hearsum [:bhearsum] from comment #0) > > This check is currently alerting if the file is 4h old, but we've discovered > > that it can be validly older than that. Can we adjust the check to wait > > until the file is 6h old before alerting? > > Ben, this check is set for 2 thresholds. Warning = 4hrs, Critical = 6hrs. > Sounds like you only want an alert when intervention is needed. Instead of > the 2 thresholds, should we just alert when the file_age breaks the 6hr mark > period? Assuming alert = IRC notications, WFM!
Alert meaning, as the alert is currently configured. If you want this check to only alert IRC, I think the Warning and Critical thresholds are fine as is. Warning @ 4hrs is a heads-up, Critical @ 6hrs means intervention is needed. members currently getting the alerts, 'buildduty' is the IRC bot 'relengapi,buildteam,buildduty,nthomas,bhearsum,coop,armenzg,catlee,asasaki,raliiev,jhopkins,hwine,jwood,catlee,coop'
(In reply to Rick Bryce [:rbryce] from comment #3) > Alert meaning, as the alert is currently configured. If you want this check > to only alert IRC, I think the Warning and Critical thresholds are fine as > is. Warning @ 4hrs is a heads-up, Critical @ 6hrs means intervention is > needed. OK, so it's actually the warning @ 4h that we want to change. We'd rather have something at 6h, and nothing before that. WARNING/CRITICAL doesn't really have a useful meaning here. > > members currently getting the alerts, 'buildduty' is the IRC bot > 'relengapi,buildteam,buildduty,nthomas,bhearsum,coop,armenzg,catlee,asasaki, > raliiev,jhopkins,hwine,jwood,catlee,coop' That looks fine.
buildbot-master65 was decommissioned and this check was moved to buildbot-master74 in Bug 987793. I've modified the thresholds so that they alert only at 6h.
Assignee: server-ops → ashish
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
(In reply to Ashish Vijayaram [:ashish] from comment #5) > buildbot-master65 was decommissioned and this check was moved to > buildbot-master74 in Bug 987793. I've modified the thresholds so that they > alert only at 6h. Thanks!
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.