we run jetstream benchmark on AWFY and would like to run it on talos as a tier-2 benchmark which we run on try/mozilla-central, just like motionmark.

this depends on a 50M patch to put jetstream in-tree: https://1drv.ms/u/s!AtgqgjhJkZQqgtVFsQEEG4F6BvfPZw

Comment on attachment 8951322 [details] [diff] [review] add jetstream to talos Review of attachment 8951322 [details] [diff] [review]: ----------------------------------------------------------------- Very cool, LGTM, and applied both patches locally and runs great. I notice that "throughput" is listed on the JetStream 1.1 test, but there's no corresponding item in talos local.json results. I'm assuming that maybe throughput is on the JetStream dashboard but they don't make it available in results maybe? r+ on confirming that "throughput" shouldn't be included in the talos results.

throughput and latency are categories related to the 49 subtests- the final score is a geometric mean (weighted a bit) of the 49 subtests.

Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/a9db2b954595 add JetStream benchmark source to tree. r=rwood https://hg.mozilla.org/integration/mozilla-inbound/rev/8508dc544f8b add JetStream benchmark to talos. r=rwood

Backed out 2 changesets (bug 1438292) for multiple failures in /builds/worker/workspace/mozharness/mozharness/base/script.py Push with failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=8508dc544f8b56d647e076e82537f4de626db089&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=runnable&selectedJob=162528182 Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=162528182&repo=mozilla-inbound&lineNumber=692 Backout: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=142b57a7b01cd14c712d00950129344e8e5fbdae&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=runnable&filter-resultStatus=success&filter-resultStatus=pending&filter-resultStatus=running

while this was backed out for infrastructure issues, we are now going to use jetstream via a tooltool artifact instead of adding 50MB to the source tree.

Well, you already landed it once and the data is forever in the repo since backouts are part of the full repo history. So if you want to land it again... ¯\_(ツ)_/¯

similar to the last patch, except this uses tooltool to download jetstream.zip. this works locally, but I think we should prioritize fixing bug 1435677 in the short term to ease the pain of running tp5, jetstream, and other webkit benchmarks locally via mach.

Comment on attachment 8953066 [details] [diff] [review] add jetstream to talos Review of attachment 8953066 [details] [diff] [review]: ----------------------------------------------------------------- Yep LGTM and I tried it out locally on OSX, runs great

Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/6f4aed8caae8 add JetStream benchmark to talos. r=rwood