web-tooling-benchmark is currently run on AWFY, primarily for v8, not for firefox. I would like to get this added for jsshell-bench right now so we have it running. If you look at what is running in awfy: https://github.com/mozilla/arewefastyet/tree/master/benchmarks/web-tooling-benchmark there is a 25MB cli.js file, we should consider making this a fetch task or some type of artifact. The source and docs are here: https://github.com/v8/web-tooling-benchmark here is the reference to running/parsing the data: https://github.com/mozilla/arewefastyet/blob/71255695f7927453778135b8b190b09b75c062fb/slave/benchmarks_shell.py#L304 I think it is just as simple as |jsshell cli.js|, then parsing the results.

Making this still up for grabs as I am busy with raptor-android atm

Add Web Tooling Benchmark & (WIP) setup fetch task

Comment on attachment 8996982 [details] Bug 1472979 - Add Web Tooling Benchmark as a jsshell-bench test I was able to make this run locally. Now I've started adding in the fetch task. I used an approach similar to :ahal's [1]. At the moment, it errors out on Try. Still, this is to know whether I'm on the right track or not. [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1472800#c21

This is the failing push to Try: https://treeherder.mozilla.org/#/jobs?repo=try&author=igoldan@mozilla.com&fromchange=bb0735e080db62dd44703e312fb7f828d003ace1

Comment on attachment 8996982 [details] Bug 1472979 - Add Web Tooling Benchmark as a jsshell-bench test Please ask someone for feedback who has experience with the jsshell framework. I haven't looked at it (yet).

I did a Try push here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a282b411874ff7c28558bf5ed254ec59d4cec8f1&filter-tier=1&filter-tier=2&filter-tier=3 All jobs are green, except the yaml linting; I fixed that also here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=017f382599a976042424e1e591900f76e9c27f51 The last revision on Phabricator now has all jobs green.

Comment on attachment 8996982 [details] Bug 1472979 - Add Web Tooling Benchmark as a jsshell-bench test Joel Maher ( :jmaher ) (UTC+2) has approved the revision.

:igoldan, lando was not able to rebase the commits, please pick this up first thing when you get back so we can get this up and running.

Pushed by igoldan@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/957f7aa848b1 Add Web Tooling Benchmark as a jsshell-bench test r=jmaher

(In reply to Joel Maher ( :jmaher ) (UTC+2) from comment #8) > :igoldan, lando was not able to rebase the commits, please pick this up > first thing when you get back so we can get this up and running. I've rebased & pushed again.