currently we upload performance data for google chrome so that it can be compared against firefox for the same benchmark. While this is useful to see how we are doing against competitors on benchmarks we want additional meta data in our graphs and dashboards. Right now we upload all data to perfherder and perfherder assumes that we have valid links to a treeherder repository, so we just take a revision. This doesn't work as we download a version of chromium and it doesn't have links to treeherder or our HG repo. I propose we do a few things: 1) eventually create a job that runs daily to get the latest builds of chromium/v8- uploading them as fetch-task artifacts 2) along side this upload (or as part of the name) determine the revision that is used for the build, so when we download and install this we have some reference to it (either filename, additional readme file as part of a .zip, a secondary artifact we download, etc.( 3) when we run a test again chromium/v8, we use this revision and pass that in with the perfherder_data object that is ingested into perfherder. In addition have a url that has the information that points to the commit history of the branch 4) we would need to update the perfherder_data schema so that it accepts these values 5) we would need to update the ingestion to handle these special cases properly- probably adding a field in the database for the revision/url 6) we would need to update the perfherder graphs and alerts so that it doesn't assume we always use revisions from hg and treeherder. This is a lot of work- for now we could print it in the log files and probably without a lot of work get it added to the json object- it is cleaning up perfherder and ingestion that could be more work. When this is done, we could support perf data from other frameworks like github/jenkins, etc. I would say the first thing to do is to have a process for simplifying the update of google products we test and ideally get that automated in a crontask.

:ahal, do you have thoughts on the first 3 steps? :wlach, do you have thoughts on the last 3 steps?

Gps had some WIP patches to get fetch tasks to support downloading directly from a github repo. So instead of needing to craft and host a URL, the fetch task would be made up of: 1. URL to the repo 2. revision 3. path within repo to file we want I think it might be a good idea to see if we can get his WIP stuff finished up and landed before spending too much time on a more robust method of updating Chromium/V8. I think this new type of fetch task would make things *much* easier. I'll see if I can dig out his patches.

chromium is not on github, or is it? We still need to solve the revision we are testing and getting that into the results we post. I agree that finishing up :gps' work on fetch tasks would give us better options here.

Hmm, I have to admit I haven't fully unpacked the treeherder/perfherder parts of the proposal here, but some parts don't feel quite right to me. For example, I really don't want to bloat the perfherder schema with revision data -- it would duplicate logic and data already in treeherder. There is no reason we can't add Treeherder repositories for Chromium/Chrome and just schedule jobs inside there -- in the past we've used Treeherder to store job information for things other than Firefox (e.g. bugzilla), and I don't see why we can't do so here. CC'ing :emorley in case he has comments/questions/feedback.