Closed Bug 1069467 Opened 10 years ago Closed 9 years ago

Need fallback import from daily jobs archive if builds-4hr is down

Categories

(Tree Management :: Treeherder: Data Ingestion, defect, P2)

defect

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: emorley, Unassigned)

References

Details

TBPL has two method of importing data: 1) For keeping up to date: builds4hr [1] 2) For initial setup (or if something went wrong with #2): using N days worth of the archived daily job results [2] If builds-4hr has a problem, or should some extra backfilling (beyond the last 4 hours) be required say in a local Vagrant instance, someone can manually run #2 specifying N. Currently treeherder only supports #1 [3], so if there was ever a problem with builds4hr [4] - we wouldn't have an easy way to fix the missing data. TBPL's implementation is at: https://hg.mozilla.org/webtools/tbpl/file/default/dataimport/import-buildbot-data.py#l297 From my understanding of the files in buildjson & also inspection of the TBPL implementation, the daily and 4hr formats are the same, so this should be pretty straight-forwards to add to: https://github.com/mozilla/treeherder-service/blob/master/treeherder/etl/buildapi.py [1] https://secure.pub.build.mozilla.org/builddata/buildjson/builds-4hr.js.gz [2] eg https://secure.pub.build.mozilla.org/builddata/buildjson/builds-2014-09-18.js.gz [3] https://github.com/mozilla/treeherder-service/blob/f4d64e72cf174233c0d39eb6f83649a6d6654a58/treeherder/settings/base.py#L223 [4] https://bugzilla.mozilla.org/showdependencytree.cgi?id=926246&maxdepth=1&hide_resolved=0
Blocks: 1080757
No longer blocks: treeherder-dev-transition
Blocks: 1088354
Priority: P3 → P2
No longer blocks: 1080757
Component: Treeherder → Treeherder: Data Ingestion
This would also help us recover from situations like those had recently, where try was missing 12+ hours worth of pushes and we had no way to re-import.
Priority: P2 → P1
Priority: P1 → P3
P2 given this would help with bug 1151806.
Priority: P3 → P2
Now that bug 1096878 is wontfix, it's less likely we'd need this. We also haven't really missed having this in the last year, so let's just wontfix. I'd rather not invest any more time than necessary into buildbot support.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.