Closed
Bug 1159700
Opened 10 years ago
Closed 9 years ago
A uniform way to find tasks for a given revision / branch / platform [/ app_name]
Categories
(Taskcluster :: General, defect)
Taskcluster
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: parkouss, Assigned: mrrrgn)
References
Details
Attachments
(1 file)
(deleted),
text/plain
|
Details |
So far I there is way to map hg commits to taskids using the index:
https://index.taskcluster.net/v1/task/buildbot.revisions.0500d91113bb.mozilla-central.linux64
So this uses the first 12 characters of the last changeset of a mercurial push.
After some discussion with :pmoore on irc, it appears that this can not be used anywhere, because it will only work for something built with buildbot (does not works for b2g for example).
Also it seems that when bug 1135206 and bug 1118394 will be landed, this won't work anymore for firefox desktop or fennec.
So basically it would be great to have a uniform way to find a task given a revision and a branch in taskcluster, :pmoore talked about something like:
taskcluster.revisions.<revision>.<branch>.<platform>
Using the index as the route to perform the mapping.
Also I have one concern (If I understand well) - the application name. I think that for example b2g and firefox desktop are on mozilla-central, so how do we distinguish those tasks ?
On ftp, this was:
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2015/01/2015-01-03-03-02-15-mozilla-central/ (firefox)
http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/2015/01/2015-01-03-16-02-13-mozilla-central/ (b2g)
Updated•10 years ago
|
Comment 1•9 years ago
|
||
Discussing with :parkouss in IRC, I think the application name is not needed explicitly in the index for buildbot builds, since the platform name is already unique. Eg: an OSX firefox build is 'macosx64', while an OSX B2G Desktop build is 'macosx64_gecko'. However, we still need to unify the index for buildbot and taskcluster, which should be handled at least in the buildbot side of things in bug 1133074.
Reporter | ||
Comment 2•9 years ago
|
||
I wrote a simple script that basically demonstrate what I need:
https://gist.github.com/parkouss/90da45fe39250f817e7f
So basically this is an attempt to get inbound builds for fennec/b2g and firefox using a changeset.
I'm currently using the route 'buildbot.*', and this does not seems to work for finding b2g builds as stated in comment #1. I suspect that b2g may not be build with buildbot anymore...
So what I basically need is a way to find artifacts from a build that occured on a given changeset for a given branch (and os/bits when this make sense). Also I find it not really useful to specify 'buildbot' in the route, in case it is built with something else the code will be broken.
Please look at the sample script, this can help to understand it.
Assignee | ||
Comment 3•9 years ago
|
||
There were a couple of issues in the construction of your route, I've fixed these here and it seems to be working. Using my revised script:
(taskcluster)vagrant@vagrant-ubuntu-trusty-64:~/90da45fe39250f817e7f$ ./get-inbound-taskcluster.py b2g linux 64
buildbot.revisions.493c86e6d74900dbd66c5c1538d7bf812fc872d7.b2g-inbound.linux64_gecko
public/build/fxos-simulator.xpi
public/build/target.crashreporter-symbols.zip
public/build/target.linux-x86_64.json
public/build/target.linux-x86_64.tar.bz2
public/build/target.tests.zip
public/logs/live.log
public/logs/live_backing.log
Attachment #8626642 -
Flags: review?
Assignee | ||
Updated•9 years ago
|
Assignee: nobody → winter2718
Assignee | ||
Comment 4•9 years ago
|
||
I found the format by looking at the task config in tree at testing/taskcluster/tasks/builds/
Then verified inspecting a task: https://tools.taskcluster.net/task-inspector/#LWHGwpQGSnujlfv4ppB7Cg/0
Reporter | ||
Comment 5•9 years ago
|
||
Oh great, thanks Morgan!
So it looks like basically for b2g the fill revision is needed (instead of the only 12 firsts characters) and also I need to add _gecko at the end.
So far, so cool - thanks for the investigation.
Now, can I use that code or is it going to break soon (because of the 'buildbot.' in the route) ? My main concern is being able to use that kind of route - even if for a some reason buildbot is replaced by something else for the build.
Assignee | ||
Comment 6•9 years ago
|
||
(In reply to Julien Pagès from comment #5)
> Oh great, thanks Morgan!
>
> So it looks like basically for b2g the fill revision is needed (instead of
> the only 12 firsts characters) and also I need to add _gecko at the end.
>
> So far, so cool - thanks for the investigation.
>
> Now, can I use that code or is it going to break soon (because of the
> 'buildbot.' in the route) ? My main concern is being able to use that kind
> of route - even if for a some reason buildbot is replaced by something else
> for the build.
That's a good point, so, I think the more sustainable route would be:
index.gecko.v1.{branch}.revision.{platform}.{rev}.{job-type}.{opt|dbg}
i.e. index.gecko.v1.mozilla-inbound.revision.linux.ba5f47ced2590d2697bca58596a36addc5d84cd7.b2g-desktop.debug
Assignee | ||
Comment 7•9 years ago
|
||
Setting the bug to resolved, re-open / tag me on another bug if there's more that your missing or this didn't solve your issue. :)
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Reporter | ||
Comment 8•9 years ago
|
||
Hm I don't think it fixes my problem completely, because if I look at a task for fennec:
https://tools.taskcluster.net/task-inspector/#H3Wb3P9MQJ-HM699X8Qnnw/
There is no other route than buildbot.*. So it seems likely to break in the future (pretty soon) when fennec won't be built with buildbot anymore (see comment 0). And same thing for firefox.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee | ||
Comment 9•9 years ago
|
||
(In reply to Julien Pagès from comment #8)
> Hm I don't think it fixes my problem completely, because if I look at a task
> for fennec:
>
> https://tools.taskcluster.net/task-inspector/#H3Wb3P9MQJ-HM699X8Qnnw/
>
> There is no other route than buildbot.*. So it seems likely to break in the
> future (pretty soon) when fennec won't be built with buildbot anymore (see
> comment 0). And same thing for firefox.
So android builds are not available yet (they are hidden in try); but they do have a route as well: index.gecko.v1.try.revision.linux.cded5316989fd54db08c5787c87b9bdbfa4c00e8.android.opt
As we port jobs to TC they will all have routes which follow this format.
Reporter | ||
Comment 10•9 years ago
|
||
Thanks, so yeah this works well!
So what can I do for now is use "buildbot.*" routes where "index.gecko.v1.*" does not work (that is, fennec and firefox)
I think that other bugs are opened to follow the migration of the route naming, so closing this bugs seems fine now.
Thanks Morgan for the help!
Status: REOPENED → RESOLVED
Closed: 9 years ago → 9 years ago
Resolution: --- → FIXED
Comment 11•9 years ago
|
||
Hey folks, I have some very recent commits to the tree that consume this indexing API over in Bug 1162191. I want folks to be aware of this consumer, which I expect to grow over time, although I'm not aware of any action needed.
Reporter | ||
Comment 12•9 years ago
|
||
Hello Morgan,
I still have some issues to find b2g builds using your revised script. I can only get linux 64 builds this way.
I tried some things, and I got this working for some platforms just by using the only 12 first characters of the changeset:
buildbot.revisions.7877051b07c4.b2g-inbound.win32_gecko
taskId: sPXqDYeWRTWZvWrYfWO6AA
buildbot.revisions.2ab4034b2cc1.b2g-inbound.macosx64_gecko
taskId: BprJO440T-Oy2jTW63O0Iw
This does the trick for mac and win 32.
I can not find any build for win 64 and linux 32 though.
Do you have an idea of how I can get those ? And is there a bug for tracking b2g route naming uniformisation ?
Flags: needinfo?(winter2718)
Assignee | ||
Comment 13•9 years ago
|
||
(In reply to Julien Pagès from comment #12)
> Hello Morgan,
>
> I still have some issues to find b2g builds using your revised script. I can
> only get linux 64 builds this way.
>
> I tried some things, and I got this working for some platforms just by using
> the only 12 first characters of the changeset:
>
> buildbot.revisions.7877051b07c4.b2g-inbound.win32_gecko
> taskId: sPXqDYeWRTWZvWrYfWO6AA
>
> buildbot.revisions.2ab4034b2cc1.b2g-inbound.macosx64_gecko
> taskId: BprJO440T-Oy2jTW63O0Iw
>
> This does the trick for mac and win 32.
>
> I can not find any build for win 64 and linux 32 though.
>
>
> Do you have an idea of how I can get those ? And is there a bug for tracking
> b2g route naming uniformisation ?
Looking at treeherder, I believe the reason you can't find them is because they don't run. There's no task tracking routes as far as I know. We could come up with a style guide for creating jobs: probably a good idea.
Flags: needinfo?(winter2718)
Updated•9 years ago
|
Component: TaskCluster → General
Product: Testing → Taskcluster
Target Milestone: --- → mozilla41
Version: unspecified → Trunk
Comment 14•9 years ago
|
||
Resetting Version and Target Milestone that accidentally got changed...
Target Milestone: mozilla41 → ---
Version: Trunk → unspecified
Assignee | ||
Updated•9 years ago
|
Attachment #8626642 -
Flags: review?
You need to log in
before you can comment on or make changes to this bug.
Description
•