From bug by Ryan: > One other interesting observation I made, though, is that automatic > backfilling/retriggering doesn't distinguish between visible and hidden jobs. > So when a new permafailing job gets turned on (happened recently) or in general > if you have a hidden permafailing job, you end up with a huge pile of > backfilled jobs that are also failing in addition to auto-retries on them when > they fail! I suspect that's a contributing factor to why we had some serious > backlog issues last week.

Hi adusca, What was the API that I could query to determine if a builder is hidden or not? I'm also considering looking at grabbing the code from here: https://github.com/chmanchester/trigger-bot/blob/master/triggerbot/tree_watcher.py#L100 It is odd to query for jobs per branch/revision when visibility is determined by repository rather than per push.

camd: is this the right way to get jobs which are hidden? I currently get an empty list. from thclient import TreeherderClient tc = TreeherderClient() # result_set_id = tc.get_resultsets('mozilla-inbound')[0]['id'] result_set_id = 21865 jobs = tc.get_jobs('mozilla-inbound', result_set_id=result_set_id, visibility='excluded') print jobs On another note, how can I *only* see hidden jobs? This shows both hidden and visible jobs: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&exclusion_profile=false

To be clear, what I'm interested is what is the list of jobs which are considered hidden on a tree (irrelevant of which push) rather than the hidden jobs of result set.

Hey Armen-- Yeah, that should be right. Here's an example of the URL doing this: https://treeherder.mozilla.org/api/project/mozilla-inbound/jobs/?count=2000&result_set_id=22039&return_type=list&visibility=excluded If you're calling that against your local instance with the client, is it possible no jobs are actually excluded for that resultset? I'll run another experiment with the client myself...

I just tested this with result_set_id of 22039 and that seems to work ok. Though I also get results from the result_set_id you used. So it seems like you may be hitting an instance that doesn't have any exclusions, rather than production?

I'm going to close a bunch of bugs and start talking with sherrifs what specifically they need. Instead of coming out with I think is needed.