Up until now SurfacePoolWayland was a pool of actual wl_surfaces, as before bug 1718569 we had no direct access to wl_buffers when using EGL. However, the way SurfacePoolCA manages native surfaces is closer to what in Wayland terminology would be a buffer pool: buffers are heavy-weight and expansive to allocate, while wl_surface objects are cheap to recreate.

So instead of having a pool of surfaces, each of them having it's own pool of buffers, make wl_surfaces part of tiles and make SurfacePoolWayland manage wl_buffers (in the form of SHM- or DMABuf buffers). This will not only allow us to share buffers more efficiently but also simplify our tile management logic.
Most importantly, we'll need to reorder wl_surfaces less often and less complex (no place_below the parent surface) and can also drop reattaching subsurfaces to compositors. Especially the former will likely decrease CPU time in compositors.

Note: Sorry in advance that this patch is so big. Unfortunately
splitting it up would create lots of redundant changes.
This should be the last big refactoring for the Wayland compositor
backend for now.

Up until now SurfacePoolWayland was a pool of actual wl_surfaces,
as before bug 1718569 we had no direct access to wl_buffers when
using EGL. However, the way SurfacePoolCA manages native surfaces
is closer to what in Wayland terminology would be a buffer pool:
buffers are heavy-weight and expansive to allocate, while
wl_surface objects are cheap to recreate.

So instead of having a pool of surfaces, each of them having its
own pool of buffers, make wl_surfaces part of tiles and make
SurfacePoolWayland manage wl_buffers (in the form of SHM- or
DMABuf buffers). This will not only allow us to share buffers more
efficiently but also simplify our tile management logic.
Most importantly, we'll need to reorder wl_surfaces less often and
less complex (no place_below the parent surface) and can also drop
reattaching subsurfaces to compositors. Especially the former will
likely decrease CPU time in compositors.

https://treeherder.mozilla.org/jobs?repo=try&revision=3fd7512db38edef2878cc789817d6add7474e306

https://treeherder.mozilla.org/jobs?repo=try&revision=ff2e2dec73dd52e4a1316f8529bc6e6957626947

https://treeherder.mozilla.org/jobs?repo=try&revision=5477835ce3ddd9a914c7c28b04915cd2e7a307db

https://treeherder.mozilla.org/jobs?repo=try&revision=0b766ce05a30954e9fb6b51a78227b3946923b7f

https://treeherder.mozilla.org/jobs?repo=try&revision=aaddcbb5d6d4bee4cd1ca6fa2061f2bb1eac0817

Latest try: https://treeherder.mozilla.org/jobs?repo=try&revision=c6f9b505266ed0387f460bf5e23b39d53685fbd5&selectedTaskRun=CBc6k_TMTOC_2H2PKXmAyg.0

Did some more intense testing and it looks very solid now, i.e. ready to go.

https://hg.mozilla.org/mozilla-central/rev/e38576ce6c5c