Once we support scale in local-space rasterization mode, we should make all off-screen surfaces use this. This will mean we can remove screen-space raster mode, and clean up quite a lot of the code that deals with selecting raster space for a given surface.

Hi Glenn, this description sounds like it's talking about RasterSpace, which we figured was unrelated to establishes_raster_root.. maybe, possibly?
Is this about removing RasterSpace? Or about removing assign_raster_roots?

My concern is that RasterSpace seemed to be something that was driven/pushed by display lists, so maybe something that Layout is trying to control, so I'm not clear how we could get rid of it purely on our side.
That said I still only understand 60% of what's going on in the code so maybe not :)

Thanks!

The terminology here is quite confusing, so it might be easier to discuss on matrix, but I'll try to clarify a bit (without confusing myself, hopefully):

• We want to support screen raster space, if the display lists requests it (e.g. the caller wants highest possible quality rendering, and doesn't expect it to animate).

• However, the semantic of the raster space in the display list is a request / hint, not a requirement. So, for example, it's reasonable that we can have a request for screen raster mode, but fall back to local raster mode (for reasons).

• The assign_raster_roots was a little bit different. Because we didn't have local-scale support, there were some cases where we couldn't draw in local-space mode, even if we wanted to (because the size of the off-screen surface would be too large).

So the end goal is that:

• We have the ability to always draw in local-space mode if we want to (having the local raster scale support gives us this).
• We can support the user requested screen raster space, but we can also choose not to, depending on other context.

One specific case where we'd like to take advantage of this is:

• If the caller requests Screen raster space, and the transforms involves perspective, we unconditionally can switch to drawing that in local-space, ignoring the request.

There are a number of benefits to doing this - but the primary win is that SWGL (the software backend Lee is working on) would no longer have to handle perspective as a general case for all primitive types - the only time it should be required is when compositing an off-screen image into the parent (so, a single specialized case).

In terms of quality, glyph rasterizers (such as freetype) can only deal with skew / rotation when drawing high quality glyphs, they are unable to deal with a perspective transform. So we don't really lose any quality from making this choice for perspective.

Thanks for elaborating Glenn!
I'm still a bit unclear if the goal here is to always force Local space for off-screen -- the title suggests so. But you also wrote

We can support the user requested screen raster space, but we can also choose not to, depending on other context.

.. with the example that for instance Lee might go in there and add a new special case for the SW path as needed.

On top of that I assumed that perspective was already always using Local space precisely because you can't use freetype to rasterize directly with an arbitrary transform.

Perhaps we could ground the task in a real world html or yaml file which currently would throw up a roadblock for SWGL; which I could then look in to for a fix? Lee, do you have a test case from your work which shows "unless the way we handle Local vs Screen is fixed, we cannot rasterize this in a reasonable way?"

Thanks!

I'll summarize a bit, more for myself than anything. But perhaps it can help clear up the terminology.

On the one hand, we have establishes_raster_root. This variable can be interpreted as "controls whether we rasterize in local space or not" thanks to this comment:

// Check if any of the surfaces can't be rasterized in local space but want to.
if raster_config.establishes_raster_root
   && (surface_rect.size.width > MAX_SURFACE_SIZE
       || surface_rect.size.height > MAX_SURFACE_SIZE)

Thanks to the local scale from Bug 1613260 this limitation is gone and so in theory we can set establishes_raster_root to true whenever we want, such as "if we have any composite mode at all" (or some other proxy for "is offscreen").
Currently, establishes_raster_root will be true if the composite mode is SVG, or the transform from surface to parent is_perspective.

Then on the other hand, there's RasterizationSpace. It comes out of DisplayItem::PushStackingContext and eventually ends up in PicturePrimitive::requested_raster_space. There is a getter that forces the space to Local(power_of_2) if the picture or its ancestor is zooming.
So always returning Local here should work in theory, but do something quite different.
In any case the value ends up in a TextRun which uses it to adjust fontsize if Some(Local(scale)).

So if "offscreen image" and "establishes raster root" have the same meaning, then perspective should already be handled fine. I'm not trying to belabour the point, I'm just trying to make sure what we're trying to do :)

Any link between the two concepts eludes me so far, I'll keep looking :)

Update after a conversation with Glenn:

The larger goal of these tasks is to make sure that a picture that wants to rasterize as a local surface, is able to do so.

Previous to Bug 1613260, a picture with a perspective transform would try to establish a raster root, but if the resulting surface was too large, it would fall out of this path. It would then revert to rasterizing inside the parent surface, and the perspective transform would be folded into all the other transforms, relying on the fact that the GPU can rasterize anything.

This would however still be incorrect, as for example a blur should happen before the perspective warp -- which won't work if we rasterize straight into the parent surface and then apply a blur post-perspective.

Thus removing the last limitations that prevent pictures from always using a raster root when they'd like to is needed for both correctness
(blur-before-warp ^1) and for backends that don't easily support arbitrary transforms -- namely software rasterization.

We think that Bug 1613260 removed the last limitation and so the goal should be achieved, paving the way for caching more surfaces, simplifying the shaders to remove fallback scenario handling, etc.

^1 on the fly scaling will still trade some visual fidelity compared to ground truth, but this is a valid trade-off and order of operations is still correct.

(In reply to Bert Peers [:bpeers] from comment #3)

Thanks for elaborating Glenn!
I'm still a bit unclear if the goal here is to always force Local space for off-screen -- the title suggests so. But you also wrote

We can support the user requested screen raster space, but we can also choose not to, depending on other context.

.. with the example that for instance Lee might go in there and add a new special case for the SW path as needed.

On top of that I assumed that perspective was already always using Local space precisely because you can't use freetype to rasterize directly with an arbitrary transform.

Perhaps we could ground the task in a real world html or yaml file which currently would throw up a roadblock for SWGL; which I could then look in to for a fix? Lee, do you have a test case from your work which shows "unless the way we handle Local vs Screen is fixed, we cannot rasterize this in a reasonable way?"

Thanks!

The goal would be just to minimize usage of shaders that would need to use perspective. Such a path in SWGL would be naturally quite a bit more expensive as I would have to do the perspective divide + near/far-plane clipping for the vertexes as well as perspective correct interpolation for the other attributes (but would probably still want to annotate those attributes with something like 'smooth' and make 'noperspective' the default to try to keep costs down). I would need to have duplicate paths in SWGL for the perspective and non-perspective cases just because the cost of perspective is so extreme. So we would want to quarantine this to only certain shaders (that are known ahead of time) and try to keep the usage of them very minimal even then.