This is an existing problem: Whenever you enter DOM fullscreen mode on a
window that has drawsContentsIntoWindowFrame == YES, we drop the information
about whether the title should be shown in the titlebar on top of Gecko's
drawing. Then, when you leave DOM fullscreen again, the freshly-created
ToolbarWindow will have mDrawTitle and titleVisibility set to their default
values: mDrawTitle defaults to NO and titleVisibility defaults to
NSWindowTitleVisible.
The title can be drawn in two different modes:

• If the ChildView is covering the titlebar and drawing it in its OpenGL
  context, the ChildView handles the drawing of the title text. That drawing
  code respects the window's mDrawTitle field, and ignores titleVisibility.
• If Cocoa is drawing the titlebar, it respects the titleVisibility property.
  At the moment, Cocoa's drawing is never visible, because it is covered up by
  the ChildView's OpenGL context. As a consequence, the extraneous title is never
  actually visible on the screen and the bug doesn't actually cause a visible
  glitch.
  Once we use CoreAnimation, Cocoa's drawing will become visible, and the wrong
  value of the titleVisibility property would become apparent.

TODO:

• Add docs for why registering / unregistering
• Remove handling of non-registered IOSurfaces, rename to UseRegisteredIOSurfaceForDefaultFramebuffer.
• Move depth buffer part into a separate patch, and also fix the ", true" in the second usage.

Depends on D26403

RenderTargetOGL::Bind on mWindowRenderTarget needs to bind the default framebuffer, not framebuffer 0.

Depends on D26404

Depends on D26405

TODO:

• fix typos in comments
• remove commented out debug logging
• compress swap chain when it exceeds 3 and surfaces in the middle become unused

Depends on D26406

TODO:

• separate out documentation for old stuff
• fix window resizing properly
• invalidateContents -> invalidateForResize

Depends on D26407

This gives us two behaviors for free which we were achieving through manual
overrides:

• The content view is sized to cover the entire window frame.
• The window controls are placed on top of the content view (instead of
  underneath it in z-order).

It also forces CoreAnimation layers for the window's entire NSView hierarchy.

NSFullSizeContentViewWindowMask is only available on 10.10 and up, but we still
support 10.9, so we cannot remove the code with the manual overrides just yet.

Depends on D26408

Depends on D26409

TODO: Only when UseCoreAnimation().

Depends on D26410

TODO: only when gfxPrefs::UseCoreAnimation().

Depends on D26411

Depends on D26404

Depends on D26623

Depends on D26405

The patches mostly work but I'm seeing some glitches during window resizes, because OMTC triggers CATransactions on the compositor thread which sometimes race with the window server's knowledge of our window shape. I have a plan to fix them but it'll take me a bit longer to implement it. The two patches in the patch stack which interact with window resizing may require some changes, so I've marked them as WIP and unset the review requests on them.

This preference doesn't do anything yet but having it at this point in the
patch stack lets me untangle things a bit.

Depends on D30402

This also makes it so that we can't tear down the ChildView while BasicCompositor compositing is happening.

Depends on D26410

TODO: This information still needs to be forwarded to WebRender.

Depends on D37915

Depends on D37916

If a composite was aborted because PreRender returned false, the composite
scheduler still resets its sense of "composite is needed". If it now receives
a flush rendering call, before this patch it would not trigger a composite.

This patch changes it so that flush rendering always triggers a composite.

I'm planning to use this as follows: During window resizing, while the main
thread is in a "being resizing" state, I want to cancel all "async" composites
and only allow "sync" / "flush rendering" composites, by returning false or
true from PreRender. This avoids overcompositing and makes resizing smoother.

Depends on D37917

Comment on attachment 9056299 [details]
Bug 1491442 - Persist wantsTitleDrawn window property across HideWindowChrome (full screen mode). r?spohl

Revision D26403 was moved to bug 1565668. Setting attachment 9056299 [details] to obsolete.

Comment on attachment 9056300 [details]
Bug 1491442 - Add support for using an IOSurface as the default framebuffer for a GLContextCGL. r?jgilbert

Revision D26404 was moved to bug 1565668. Setting attachment 9056300 [details] to obsolete.

Comment on attachment 9056737 [details]
Bug 1491442 - Create a depth buffer for the default framebuffer of a GLContext that is used with WebRender. r?jgilbert

Revision D26623 was moved to bug 1565668. Setting attachment 9056737 [details] to obsolete.

Comment on attachment 9056738 [details]
Bug 1491442 - Make WebRender draw into the default framebuffer. r?mattwoodrow

Revision D26624 was moved to bug 1565668. Setting attachment 9056738 [details] to obsolete.

Comment on attachment 9056301 [details]
Bug 1491442 - Read screenshots from the correct framebuffer when the default framebuffer is overridden. r?mattwoodrow

Revision D26405 was moved to bug 1565668. Setting attachment 9056301 [details] to obsolete.

Comment on attachment 9063608 [details]
Bug 1491442 - Add CFTypeRefPtr which is a RefPtr-style smart pointer for CFTypeRef types. r?jrmuizel

Revision D30402 was moved to bug 1565668. Setting attachment 9063608 [details] to obsolete.

Comment on attachment 9077764 [details]
Bug 1491442 - Add an off-by-default preference called gfx.core-animation.enabled. r?jrmuizel

Revision D37914 was moved to bug 1565668. Setting attachment 9077764 [details] to obsolete.

Comment on attachment 9056307 [details]
Bug 1491442 - Add back-pressure to CompositorOGL. r?sotaro

Revision D26411 was moved to bug 1565668. Setting attachment 9056307 [details] to obsolete.

Comment on attachment 9056308 [details]
Bug 1491442 - Add back-pressure to WebRender+OGL by implementing RenderCompositorOGL::WaitForGPU(). r?sotaro

Revision D26412 was moved to bug 1565668. Setting attachment 9056308 [details] to obsolete.

Comment on attachment 9056306 [details]
Bug 1565668 - Do not override _wantsFloatingTitlebar when using CoreAnimation. r?spohl

Revision D26410 was moved to bug 1565668. Setting attachment 9056306 [details] to obsolete.

Comment on attachment 9077765 [details]
Bug 1491442 - Make BasicCompositor PreRender/PostRender handling more consistent with CompositorOGL / WebRender. r?mattwoodrow

Revision D37915 was moved to bug 1565668. Setting attachment 9077765 [details] to obsolete.

Depends on D37920

Depends on D38747

BasicLayers is main thread drawing. BasicCompositor is compositor-thread drawing.

Depends on D38748

Depends on D38749

Depends on D38750

We always use OMTC when using OpenGL. We also currently always use OpenGL when using OMTC, but that's about to change.

Depends on D38751

Unlike what the old comment in its drawRect method says, this isn't actually
dependent on the NSFullSizeContentViewWindowMask. Any window that uses
CoreAnimation layers for its window frame will apply these effects automatically.

Depends on D38752

This patch leaves you with empty windows everywhere. We will build the new
rendering paths from the ground up in the upcoming patches.

Depends on D38753

This makes us host an empty layer on mPixelHostingView. It also causes all windows
(including context menus, tooltips, arrow panels etc.) to use CoreAnimation layers
for the window frame. This is achieved by calling setWantsLayer:YES on every
window's content view.

After this changeset, all windows will still be empty.

Depends on D38754

This makes context menus work. Regular windows are still blank at this point.

Depends on D26407

This makes windows that render using CompositorOGL or WebRender show content.

Depends on D38756

Now CoreAnimation supports all rendering paths.

Depends on D38757

There are three cases here: Window opening, window resizing, and window
activeness state change.
Each one of these triggers a CoreAnimation transaction on the main thread. By
default, those transactions do not call our layer's display method because the
layer is not marked as needing display. Usually, what this means is that the
window will be painted twice: Once in the main thread transaction, and then
another time on the compositor thread once Gecko has noticed a state change and
triggered its own composite in response. This can look glitchy: For window
activeness changes in windows with titlebars, the titlebar would change color
at a different time than the content. And for window resizing it would look
even worse.
Calling markLayerForDisplay will result in a call to displayLayer in the
upcoming CoreAnimation transaction and lets us update the entire window in one
atomic paint.

Depends on D38758

Window overlay drawing was added as a workaround for the following:
When our NSOpenGLContext covered the entire window, it would cover the titlebar
contents and hide the window buttons and the title string. It would also not
get anti-aliased rounded corner clipping.

In windows that use CoreAnimation layers for the window frame, this is no longer
a problem, because the CoreAnimation layer tree takes care of these effects:
It applies rounded corner clipping to the window content layers, it puts the
window buttons on top, and it also puts the title string on top if it is shown.

So when we're using CoreAnimation, the existing code needs to be deactivated,
otherwise we'd draw those things twice.

Depends on D38759

This is only a temporary solution. In the near future, we want to split up this
layer into opaque and non-opaque sublayers so that we can have both: vibrancy
in the places that should be vibrant, and efficient compositing in the places
that are actually opaque.

Depends on D38760

Depends on D40513

This makes context menus work. Regular windows are still blank at this point.

This introduces a visual regression on 10.9: context menus and panels now no
longer have a shadow. Only 10.10 and above support shadows on transparent windows
that use CoreAnimation; 10.9 is not able to obtain the shadow shape on those
types of windows.
I think this is an acceptable regression to take. We want to use CoreAnimation
for all window types because it simplifies the code (no need to handle two
paths) and because it avoids expensive mode switches if we realize too late
that a window we just opened is supposed to use CoreAnimation.

Depends on D26407

This makes windows that render using CompositorOGL or WebRender show content.

Depends on D40516

This avoids most window resizing glitches.

Depends on D38758

Doing so causes GL_INVALID_OPERATION errors:
[gl:0x121a59000] void mozilla::gl::GLContext::fReadBuffer(GLenum): Generated unexpected GL_INVALID_OPERATION error.

Depends on D40550

The patches are fully ready for review now. Unfortunately Bugzilla doesn't display them in the right order, so I recommend looking at the "Stack" view on phabricator to see the correct order.

Try pushes:
CoreAnimation off: https://treeherder.mozilla.org/#/jobs?repo=try&revision=18bc2fd6a4700040643ec22a0661ca2fc593cbe4
CoreAnimation on: https://treeherder.mozilla.org/#/jobs?repo=try&revision=269fd0f81f44a6a97980cb89947c3d2edf116b9e

Both are green except for some intermittents.

As with comment 10, I don't see any GPU power saving with the Core Animation enabled Try build compared with Nightly using the default vibrant window.

1280x800, WebRender OFF, MacBookPro11,1, speed scroll mozilla.org up & down
Nightly, opaque GL context ~4 W GPU
Nightly, transparent GL context ~12 W GPU
CA enabled Try build, sidebar collapsed ~12 W GPU

Chrome --disable-mac-overlays ~4 W GPU

The Core Animation enabled Try build seems to show all the correct behaviors in Quartz Debug eg opaque (green) main content window, vibrant (red) sidebar & tab bar.

Thanks for testing; this is expected. The work in this bug gives us an equivalent implementation to what we had before, but using CoreAnimation. There are no power savings yet. The power savings will come from the work in the follow-up bugs; see the "Depends on" list in bug 1429522 for the full list. (If only it was as simple as flipping a switch... Saving power with CoreAnimation is all about how you use CoreAnimation: How many CoreAnimation layers do you create, how do you distribute your window's contents into these layers, how many of the layers are opaque, how large are the layers that you touch when anything changes, how many changes can you handle by simply moving layers and not repainting their contents, etc. Chrome has done a lot of work to create layers in smart ways, whereas the work in this bug is just the simplest thing that works, because it's only the first step: One transparent layer that covers the entire window.)

A quick note on Quartz Debug, though: Unfortunately, for windows that use CoreAnimation, Quartz Debug's opaque color tinting only works on the window as a whole, and doesn't tint individual layers within the window. So it tints the window green even if the window contains transparent layers in front, on top of the window's opaque background layer. For a more accurate idea of opaque and transparent layers, you can start the Firefox process with the environment variables CA_LAYER_SURFACE=0 CA_COLOR_OPAQUE=1: This will tint the layers within the window. But be aware that doing so will change power consumption characteristics because CA_LAYER_SURFACE=0 causes in-process layer compositing as opposed to the usual out-of-process, within-WindowServer CoreAnimation layer compositing.

Let's keep this bug focused on landing the patches. Once the work to actually save power is further along, I will let you know.

Great. Thanks for the detailed explanation. I'll shut up now :)

https://hg.mozilla.org/mozilla-central/rev/fb949ba38402
https://hg.mozilla.org/mozilla-central/rev/38705780d7f8
https://hg.mozilla.org/mozilla-central/rev/9dcce9060dac
https://hg.mozilla.org/mozilla-central/rev/82b63cd1a2d3
https://hg.mozilla.org/mozilla-central/rev/9216803cdf01
https://hg.mozilla.org/mozilla-central/rev/d66f54731eeb
https://hg.mozilla.org/mozilla-central/rev/8e82722106c0
https://hg.mozilla.org/mozilla-central/rev/9ffcdc648de5
https://hg.mozilla.org/mozilla-central/rev/e89c3a6b2cd8
https://hg.mozilla.org/mozilla-central/rev/372c720b6ce8
https://hg.mozilla.org/mozilla-central/rev/cf2547ffbfd4
https://hg.mozilla.org/mozilla-central/rev/7711755e979e
https://hg.mozilla.org/mozilla-central/rev/5aa5080461f8
https://hg.mozilla.org/mozilla-central/rev/24dbae685801
https://hg.mozilla.org/mozilla-central/rev/a5ba77776417
https://hg.mozilla.org/mozilla-central/rev/e6e88a4fc6b9
https://hg.mozilla.org/mozilla-central/rev/706ff8a1badc
https://hg.mozilla.org/mozilla-central/rev/ae86a277bd72
https://hg.mozilla.org/mozilla-central/rev/0822612163d6
https://hg.mozilla.org/mozilla-central/rev/03a1e5d857b5