Rendering masks

We have optimizations to avoid rendering very large masks, for example by building nine-patches. When masks become a bit more complicated, though we end up just building a single contiguous mask which can be expensive.

To address that we could introduce a notion of tiled mask: the mask is split into a regular grid and we detect on the CPU which tiles are fully masked and which tiles are not masked at all. This would let us avoid worst cases with large complex masks (with several rounded rects for example).

Figuring out the tiling scheme and detecting fully transparent/opaque tiles isn't very hard but the details of how to integrate it into webrender are probably going to be subtle. Ideally we wouldn't have a extra branch or shader variation in all brush shaders. Instead the frame builder would emmit quads with information about where to sample the mask without needing any context about whether the mask is part of a nine-patch or a grid.

Getting closer to rendering some parts of SVG

The other motivation for this is to gradually move webrender closer to being able to handle SVG paths efficiently. With the ability to rasterize masks and render various types of brushes (images, gradients, etc.) with these masks we technically have what it needs to render some SVG. However if we rasterize a contiguous mask for each path on the CPU, the amount of GPU memory required and the amount of data to upload qucikly becomes problematic for a typical SVG drawings, which are built upon many large shapes stacked on top of one another.

Splitting masks into regular tile grids is an effective way to save storage and upload for all tiles that are fully masked or fully visible and make storage and uploads manageable. In addition, the grid provides a very easy coarse occlusion culling scheme to avoid overdraw and even avoid rasterizing masks for hidden tiles.

There is a some prior art around the theme of using regular grids for vector graphics:

• The "Massively parallel vector graphics" paper being one of the first http://w3.impa.br/~diego/projects/GanEtAl14/ that I know of
• Raph Levien's piet See https://raphlinus.github.io/rust/graphics/gpu/2019/05/08/modern-2d.html and other posts on the same blog
• pcwalton's pathfinder, See this blog post explaining how it works: https://nical.github.io/posts/a-look-at-pathfinder.html Note that since the post was written patfinder added code paths to a do few things similar to piet on some hardware.

In a nutshell the various steps of rendering with a regular grid are:

• Assigning content to tiles. There are simple ways to do this on the CPU, advanced ways to do it on GPU, I would start with doing it on CPU
• Determining coverage, which can be either an extra step as in pathfinder or done during compositing as in the RAVG paper and piet. Some falvors of it can also be done on the CPU or the GPU (again, balance between simplicity and efficiency).
• Compositing, in which the color buffer is written, either by sampling masks or while doing determining coverage.

Interestingly, if tile assignment, coverage and compositing are separate steps as in pathfinder, there is a lot of flexibility and room for changing parts of the algorithm without the others. In our case the compositing could probably be performed by brush shaders without big changes. These steps can also have simple implementations on old/buggy/low-end hardware and advanced faster implementations on high end hardware without changing much of the infrastructure.

A good starting point would be to converge slowly towards the model described in pathfinder's blog. The details don't have to be the same initially, we can start with a simple 8bit integer mask format with masks rasterized on the CPU, and add fast paths involving compute shaders (wherever available) in the future.

The occlusion culling aspect adds a lot of wins so it would make sense to use the same coordinate system for the tile grid of all masks in the same animated geometry root and take advantage of knowing about fully opaque tiles across primitives.