Bug 674731 has shown that we need performance regression tests for WebGL. Do we have a framework for performance tests that we could reuse for that? Is Talos what we want? Any wiki page explaining how to get started with that?

Doesn't Marc Cote have a tool for this in progress?

Afaik this was for the microsoft demos

It is indeed being used for the MS demos at the moment, but it could potentially be used for any JavaScript-based tests that can provide relatively simple test results (e.g. fps). I tried to make it pretty generic. I haven't had a chance to work on it for a little while, but I intend to get back to it and add more tests before too long.

Talking to Ehsan, we should use Talos tests for this. This should be relatively high-priority especially in light of bug 726396 and bug 724476.

This should be our highest priority once we're done with fixing current regressions and fixing 1.0.1 conformance (hopefully in a few days). But, everyone seems to have a mystical fear of Talos. Maybe because of "Eros and Talos" ? anyway. Let's file a separate bug for first writing performance tests as pain web pages. This will presumably get done much faster, will be generally useful, and once it's done the Talos part will be doable by anyone without WebGL knowledge.

FWIW, the talos part is very easy. You basically just call a js function with a time value. Still, standalone performance tests are a very good idea too.

Facebook's jsgamebench includes a webglbench directory. Whether it's even slightly possible that it's suitable for this I haven't checked, but it could be worth a look.

WebGL performance regression tests are here: http://hg.mozilla.org/users/bjacob_mozilla.com/webgl-perf-tests

Going to take a stab at this, using Talos. Hard thing is that some tests don't really report a value, they just show a value while running (e.g. fps for the MS demos).

Are you trying to get detailed performance information, or just what the test measures, e.g. fps? If it's the latter, my SpeedTests framework would probably be simpler for this--although at the moment it is only run twice a day. I could conceivably make this continuous. Just curious as to what your exact requirements are. I am plotting results from various tests at http://mrcote.info/speedtests/results.html alongside those from other browsers.

Well, talos already includes things like getting memory/cpu/etc. info which would be nice to have, but the core need is just a single benchmark result per test. SpeedTests seems to do that, but how hard is that to deploy across different OS/machines? We'd need to cover at the very least Win7, OSX, Android, B2G.

(In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #9) > Going to take a stab at this, using Talos. Hard thing is that some tests > don't really report a value, they just show a value while running (e.g. fps > for the MS demos). Hey, don't hijack this bug for general performance tests! This was only about WebGL performance regression tests. The tests to be integrated into talos are http://hg.mozilla.org/users/bjacob_mozilla.com/webgl-perf-tests

Not hijacking, I assumed it was for general webgl regression tests, not specifically for yours :) Either way, we should look into whether integrating it into talos or into speedtests is better..

Sure, not specifically mine --- but this should remain WebGL focused; I was referring to your mention of MS tests: (In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #9) > Going to take a stab at this, using Talos. Hard thing is that some tests > don't really report a value, they just show a value while running (e.g. fps > for the MS demos). ^ this

Yeah, speedtests does not gather data about memory, cpu, etc. At the moment it is only running on Windows 7. OS X would not be too difficult. Android is planned but relies on some other infrastructure first. So if you are looking to get this up on all platforms asap, speedtests may not be the way, although I have no idea how much work it would be to get it going on talos.

(In reply to Benoit Jacob [:bjacob] from comment #14) > Sure, not specifically mine --- but this should remain WebGL focused; I was > referring to your mention of MS tests: Pretend I said "Google WebGL samples" then :D