For TabCandy, we need the ability to scale images quickly, sometimes many at a time (20 being a probable upper bound). Even using CSS Transitions animation of image scaling is too slow to create an acceptable user experience. In general, animating images is a not uncommon action for websites. As we get increasingly rich web-apps, we'll see more and more use-cases that require this (think photo galleries where an image is zoomed into, or games, or...). A paired down example of just how slow scaling images can get is available here: http://azarask.in/projects/tabcandy/scale/css-animation.html

Have you tried testing on a recent trunk Windows build with D2D enabled? See http://www.basschouten.com/blog1.php/2010/03/02/presenting-direct2d-hardware-acceleratio [sic, no trailing n!]

We should probably layerize images that are undergoing size transitions. Then the accelerated layers backends will help us as well. Also, vlad's canvas-layers patches make canvas scaling accelerated with an accelerated layers backend.

For people without usable GPUs, I don't know how good pixman's SSE scaler is these days. Jeff might know.

For what it's worth, I've tested it with Direct2D and it's very smooth.

@Dolske: I've only been playing with this on OSX but am excited to hear that the Direct2D build made it smooth. I'll try to find a machine to play with it on. @roc: Are there any plans for those fixes to make there way into the Firefox 4 beta?

As a short term thing, using "image-rendering: -moz-crisp-edges;" in the CSS seems to help a bit on OS X (although "image-rendering: optimizeSpeed;", ironically, doesn't.)

Seems to me that if we accelerate the tabcandy window (add accelerated='11' to the root element), apply vlad's patches to use layers for canvases, and use canvases directly instead of copying the contents to images, then this would be really fast everywhere we have an accelerated layers backend (GL) *or* accelerated cairo rendering (D2D). For Firefox 4, the plan is to support having every window accelerated as the hardware/drivers allow, so that first requirement would go away. The plan is also to have a GL layers backend that works for Mac, Linux and Windows users with reasonably capable GPUs and GL drivers (i.e., not the Intel onboard stuff). That leaves two sets of people getting a crappy experience: 1) people with no usable GPU 2) people with a GPU that can only be used by D3D9 (no D2D) For the latter, we really need a D3D9 layers backend. For the former, we need to profile this testcase, optimize where we can, and degrade gracefully.

Oh one more thing: does this use CSS transforms to scale/position the canvases? If so, that won't be accelerated by layers, currently. Only setting the CSS width/height will give you accelerated scaling, and it won't work if it's inside a CSS transform. Accelerating CSS transforms is not on the schedule for Firefox 4.

re comment 8: Hm. That's a bit of a snag. Is there any way to get the animation effects in a way that's accelerated?

Yes, use absolute positioning instead of CSS transforms.

After a quick chat with Vlad, Aza and I think we have the list of what's planned to be accelerated for Firefox 4: - using CSS transitions to modify width and height attributes - using CSS transitions to modify element position - using CSS transitions to modify opacity But not: - using CSS transforms to modify size of elements - using CSS transforms to rotate elements That right?

(In reply to comment #11) > After a quick chat with Vlad, Aza and I think we have the list of what's > planned to be accelerated for Firefox 4: > > - using CSS transitions to modify width and height attributes > - using CSS transitions to modify element position Yes, for canvases. Setting the position doesn't really need to be accelerated. > - using CSS transitions to modify opacity Yes for everything. > - using CSS transforms to modify size of elements > - using CSS transforms to rotate elements > > That right? Yeah. It wouldn't be terribly hard to accelerate CSS transforms if we needed to, just extra work that'd be better kept out of the Firefox 4 schedule. Definitely want to do it as soon as we can get to it.

(In reply to comment #12) > (In reply to comment #11) > > After a quick chat with Vlad, Aza and I think we have the list of what's > > planned to be accelerated for Firefox 4: > > > > - using CSS transitions to modify width and height attributes > > - using CSS transitions to modify element position > > Yes, for canvases. Setting the position doesn't really need to be accelerated. For canvases only - Aza; do you guys retain the objects as canvases, or do you use canvas to draw them as an image and then act on that? (thanks, roc!)

We retain them as canvases. Canvases draw just fine and there is no need to go through the costly toDataUrl step. My understanding from Vlad is that in the future we may put images that are being CSS transitioned into their own layer for acceleration, but because we do all of our heavy hitting in TabCandy with canvases, we are golden. @vlad/roc: Is there a way to get a build with Vlad's patches so that we can try out your suggestions?

(In reply to comment #13) > For canvases only - Aza; do you guys retain the objects as canvases, or do you > use canvas to draw them as an image and then act on that? The thumbnails in our tab representations are just canvas objects; we don't convert them to images. All of the styling around the thumbnail for each tab is css, however. Would it improve speed if we rendered each tab (Thumbnail + styling + title, etc) as a canvas?

If you're scaling the styling and the title, yes. Otherwise no.

I implemented the accelerated=11 attribute and changed the objects being scaled to canvases, yet still when scaled via CSS transitions the animation stutters. http://azarask.in/projects/tabcandy/scale/css-animation.html#20-canvas My understanding was that Vlad's patch had landed, meaning that canvases under transforms should be accelerated?

To get acceleration, you'll have to load it as a toplevel window, not as a content document. If you look in the Error Console you should see a message saying a GL or D3D Layer Manager is being used.

Does this mean that content won't be able to get the benefit of hardware acceleration of canvas being CSS transitioned?

It will when bug 130078 is fixed.

My make-go-faster extension can help you test this: http://people.mozilla.com/~vladimir/misc/makegofaster.xpi Open up tabcandy, then go to the tools menu and select Faster. The window that it opens is accelerated.

(In reply to comment #21) Vlad: I love the name, but every time I try to use the extension I get the following error. Error: makegofaster is not defined Source File: chrome://browser/content/browser.xul Line: 1

(In reply to comment #22) Nice.

Vladimir, I've tried you extension (installed, click the checkbox) on Windows XP and seems to me Tab Candy is still very slow. Latest daily build (Minefield) and latest DX9, ATI Radeon 9550 128MB

Graphics accelaration provided by D2D is not available on XP, and therefor the TabCandy stuff should not require it.

(In reply to comment #25) > Graphics accelaration provided by D2D is not available on XP, and therefor the > TabCandy stuff should not require it. I thought the graphics acceleration was being provided by D3D9 accelerated layers.

It is, yes. Though as far as d2d goes, XP is on it's way out. We should work fine on XP, but we're not going to limit ourselves to "must work as fast on XP as on other platforms".

May be not precise 'as fast on' but still, Win XP is more than 50% according to: http://www.w3schools.com/browsers/browsers_os.asp "Windows XP is the most popular operating system." "2010 Win7 Vista Win2003 WinXP W2000 Linux Mac July 20.6% 10.9% 1.3% 54.6% 0.4% 4.8% 6.5%" Don't want to block exiting new developments like layering, tabcandy, etc, but more than 50% is still using WinXP, and keep on doing this for some time, looking at the trends.

OK, so when D3D9 lands, tabcandy effects should be faster on XP, is what I believe Vlad is saying in comment 27 - correct?

If it is not faster, it will be possible to make it faster, yes.

Apple even made their Video Bitsteams lower demanding to reach as much user as possible on their platform, not supporting Win XP D3D9 would be a disaster for many old Netbooks and such they couldn't use Tab Candy as fluid as others and investing into Win7 or Vista seems not really a solution for them :(

PS: Though they could switch to Linux with better working OpenGL support i guess :P any benchmarks of the difference between the current D3D9 and OpenGL based layers on Win XP ?

(In reply to comment #32) > PS: Though they could switch to Linux with better working OpenGL support i > guess :P > any benchmarks of the difference between the current D3D9 and OpenGL based > layers on Win XP ? Currently D3D9 is -much- faster, but that's really just because it's more complete than the OpenGL one I think. In general the performance difference I expect won't be -too- big on drivers where openGL is well supported, although it'll be tricky to make it as fast on OGL as it is on D3D9, GPU memcpy for example is slightly harder on OGL (but there is an extension that makes it possible though!) In any case as a lot of XP users are also on Intel hardware where OpenGL is poorly supported. In any case, you can test on windows XP how it runs with D3D9 layers by specifying the 'MOZ_ACCELERATED=11' environment variable.

thx for the enable hint :) seems to work @ least i see a first difference when opening a new tab (it's being rendered black @ first with the systemvariable set) This though didn't fix my problem with the Fullscreen Video Playback i described here https://bugzilla.mozilla.org/show_bug.cgi?id=589357

actually i see the same as in the Video Fullscreen Playback happening now on canvas test pages like Microsofts Fish Tank Demo (the truncated stuff in this case is from a 3D Game (D3D9) running in the background, seems to be exactly the same problem that is described in the https://bugzilla.mozilla.org/show_bug.cgi?id=586796 bug report for Mac OSX. http://img840.imageshack.us/i/layeraccelerationproble.png/ <- what im experiencing seems all of those bugs are the same problem in the layer acceleration ?

with it enabled tab candy now shows only a black background (nothing gets rendered @ all) http://img52.imageshack.us/i/tabcandylayerfailsxp.png/

This will be fixed when bug 589634 lands.

Note: bug 589634 seems to have landed, but our zoom in/out animation still feels slow. Should this bug become Component=TabCandy for continued discussion of our effects?

The zoom animation seems to happen in chunks. Particularily switching back to normal view, the selected tab image expands to halfway, and then stops, and then the fullscreen.

With the latest b5, I just tried going to the test page <http://azarask.in/projects/tabcandy/scale/css-animation.html#100-canvas> and the animation is still as chunky as ever despite bug 130078 being fixed.

To make that testcase great, we need to run the browser window using an accelerated layer manager. For Windows, the D3D layer manager is close to ready, but GL (for Mac and Linux) is less good at the moment. To see how well that stuff works on each platform, run with the MOZ_ACCELERATED=11 environment variable set.

(Comment #37 referred to a bug that made TabCandy performance suck *even if* MOZ_ACCELERATED=11 was set.)

re: comment 41 - from today's development meeting, "The expectation that fixing bug 130078 would cause the animations to be faster was misplaced. It is a step in the right direction." Hopefully the speaker of thus, jrmuizel, can expand here on what will actually make animation go fast now please thanks.

(In reply to comment #44) > Hopefully the speaker of thus, jrmuizel, can expand > here on what will actually make animation go fast now please thanks. Using an accelerated layer manager, as roc mentioned, should make it go fast.

Which is easy to test -- run on windows with MOZ_ACCELERATED=1 in the environment and see what happens.

After talking with joedrew, my understanding is that as roc says we need to have MOZ-ACCELERATED=11 turned on to get the benefits of acceleration. For this to be useful for Tab Candy as a feature, it needs to be turned on by default. This will happen in a platform-dependent way. In order of which will happen first: Windows (Vista & XP): Bug 581212 Mac: Bug 580405

Note that you don't even need to mess around with environment variables - just set the about:config pref layers.accelerate-all to true.

Is there a Linux bug as well?

afaik, no plan on enabling GL layers by default on linux right now.

Bug 580405 technically applies to Linux as well.

On Linux, if your X driver accelerates XRender well (as modern NVidia binary drivers seem to do, for example), we should already be fast without enabling GL.

It doesn't got better on XP yet still slow to scale the main tab group even with D3D9 on :( The cpu utilization is insanely high 1 entire core gets utilized but no smooth animation if you try to scale fast though if you move the mouse slowly it works fine but as soon as you move it faster the whole CPU is eaten up by the redrawing of it :(

Maybe I'm missing something, but can't you try rendering ahead? Last time I checked the scaling was more of a bottleneck than the blitting, so this should let you animate smoothly and the delay should be acceptable if the animation is short (10 frames or so). Memory usage would spike for a short moment, but given an average machine with a non-acceleratable graphics adapter I wouldn't expect more than 1280x1024 pixels, meaning (if the initial rectangle is 10% and 10 frames are sufficient): fullFrame=1280*1024*3bytes fullFrame*(.1+.19+.28+.37+.46+.55+.64+.73+.82+.91+1)==FF*6.05 meaning less than 23MB, even if the whole screen was covered at 1280x1024. Or 36MB for FullHD.

Another idea: Does the software nearest-neighbor scaler posess a fast path for integer (2x, 3x, 4x) scaling? If not it should be RELATIVELY easy to add and TabCandy could simply render the animation at the a lower resolution (say, 256x150 for 1024x600 area) and then scale it up in a second pass using the fast scaler.

Runs smooth like silk on my little Netbook (Intel, no acceleration, 1024x600 full screen).

Just tried it on the big screen (1680x1050) without D2D and accelerate-none (which I think is pretty much the worst case scenario). Works pretty well there too.

Hans, these are all great questions and a great exploration. A couple questions: (1) My original use case runs pretty smoothly with only one image, when it got up to 20 it started getting chunky. Can you try your patch but with 20 images? (2) It would be awesome to see what happens when you integrate this into Tab Candy. Do the results work in context? (3) I was thinking about the same thing re pre-caching. Not sure how far in advance we would have to guess on which tab was going to be zoomed. Would you mind playing with it?

It appears that on the latest builds in Firefox, this animation is now smooth.

(In reply to comment #59) > It appears that on the latest builds in Firefox, this animation is now smooth. Hmm. It appears to be improved but still seems to stop slightly around the middle of the zoom-in. Zoom-out (normal -> panorama) does not show ANY animation most of the time. Sometimes however it does.

Michael, you're on Linux right? Performance here is going to depend heavily on platform-specific code, especially whether GL/D3D/D2D are enabled.

(In reply to comment #61) > Michael, you're on Linux right? Correct. > Performance here is going to depend heavily on > platform-specific code, especially whether GL/D3D/D2D are enabled. Last time I tried OGL (not too long ago) it seemed to slow everything down. Did this improve recently (or will it for 4.0?). The OGL bugs I am subscribed to seem to be very quiet. I tested this on a system with GeForce 7950 and one running Intel x3100 and get about the same experience on both systems.

Right now the focus for GL is Android and Mac. I don't know if we're going to be able to do much for desktop Linux specifically in the 4.0 timeframe, but some of the optimizations definitely help all GL users. Keep trying it! Having said that, the hardware/driver situation on Linux is even more spotty than elsewhere :-(.

I can see the zoom-in jitter Michael described in comment #59 on a Mac.