Closed Bug 685299 Opened 13 years ago Closed 13 years ago

Need ftp log uploading for projects, Valgrind and SpiderMonkey builds

Categories

(Release Engineering :: General, defect, P3)

x86
All
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: lsblakk, Assigned: catlee)

References

Details

(Whiteboard: [automation][ftp][database][projects][logs])

Log uploading for projects will be important for getting away from Tinderbox. This has come up in Jetpack error log parsing but it would be great to get this for any and all projects. Here's a list of what is needed to get this happening (until a blob store comes along): - set an ftp_url & path in configs for each project* - append a status/log_handler to the generateProjectObjects buildOjects['status'] that can handle log uploads - create a unique ID for each project build and set it as a property - use this for uploading and insert into db with builder results - log_uploader and post_upload need to be taught about projects - tbpl integration (handwaving here, not sure what's involved) * anything that's it's own project should have it's own dir on ftp (Spidermonkey, Nanojit, Jetpack) but need to accommodate a project working on different branches.
Is bug 684154 a project-specific version of this ?
Blocks: 684159
Blocks: 625979
No longer blocks: 684159
Blocks: 684159
No longer blocks: 684159
Blocks: 630538
No longer blocks: 625979
Questions Have Been Raised about why this blocks the tracker we're misusing as "stop sending Firefox and related tree mail to Tinderbox" so adding the "projects" that are actually builds on Firefox and mozilla-inbound to the summary.
Summary: Need ftp log uploading for projects → Need ftp log uploading for projects, Valgrind and SpiderMonkey builds
Blocks: 625979
Blocks: 709203
Best choice for integrating with tbpl: partly-fix bug 611448 by teaching tbpl (around http://hg.mozilla.org/users/mstange_themasta.com/tinderboxpushlog/file/5128409cbd2a/dataimport/import-buildbot-data.py#l106) that if there's a logURL property, it should use that, and then stick wherever you're going to upload them in that property. Otherwise, reverse engineering tbpl's reverse engineering of buildbot, you need to set the properties: product, platform or stage_platform, buildid, and buildnumber, and then stick the logs in http://ftp.mozilla.org/pub/mozilla.org/$product/tinderbox-builds/$branch_platform/$buildid/$builder-build$buildnumber.txt.gz where the funky bits are that $branch_platform comes from the branch property + "-" + the platform property, $buildid (unless it changes before you get to this) is where you stick 20111212173719 in the buildid property and tbpl converts it to epoch seconds, and if bug 671450 lands and you follow it and it doesn't change tbpl, you need to symlink that epoch seconds dir to the real dated dir, or, handwaving yadda... Doesn't a logURL property sound nice?
per meeting with catlee, he's doing some of this in bug#713846. Dont have enough bugzilla-fu to know whether that means its "depends on" or "blocks" or "DUP", so picking one for now just to link them together for future reference.
Depends on: 713846
Assignee: lsblakk → catlee
Whiteboard: [automation][ftp][database][projects] → [automation][ftp][database][projects][logs]
Status: ASSIGNED → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.