Problem: The dynamic toolbar and compositor operate in different threads. Each thread is responsible for drawing its content synchronized with the other. As the toolbar and surface are moved in the UI Java thread, the compositor must adjust the content rendered to the surface so that the entire exposed surface is filled. In order for the two threads to remain synchronized, the frame metrics from the compositor must be shared with the UI thread every frame and the UI thread provides the fixed frame margins. This is accomplished by calling nsWidget::SyncFrameMetrics() from the compositor thread. This works as long as both threads reside in the same process. With the addition of the GPU process on Android, they no longer exist in the same process. A new solution needs to be found to enable the dynamic toolbar to function smoothly across the two processes. Proposed Solution: Currently the toolbar animation is handled by the Java class DynamicToolbarAnimator. All UI element placement is handled by an instance of this class. Moving the animation of the toolbar to the compositor thread would eliminate the need to synchronize the UI thread with the compositor thread. How: A new C++ version of the DynamicToolbarAnimator class would be created. It would be instantiated and live in the compositor thread. The live toolbar UI elements would still live in the UI thread. However, instead of animating the actual UI elements, the UI thread would only manage the binary state of visible and not visible. Instead of animating the toolbar UI elements, the C++ DynamicToolbarAnimator would animate a static snapshot of the toolbar that is shared from the UI thread to the compositor thread via IPC. This is possible because the toolbar should not visually change while it is being animated. An additional change would be in the placement of the surface view. Currently the surface is created with the same dimensions as the screen and then shifted down to accommodate the toolbar when it is visible. The fixed frame margin ensures that fixed position elements and scrollbars appear in the proper location. By moving the toolbar animator to the compositor thread, it would no longer be necessary to shift the surface. Instead, the Toolbar UI elements would be rendered on top of the surface and the compositor would render to the portion of surface that is not obstructed by the toolbar. Hiding the toolbar would work as follows: 1) Content is scrolled so the UI thread passes a snapshot of the toolbar to the compositor. 2) The compositor starts rendering the snapshot underneath the UI toolbar. 3) The compositor thread notifies the UI thread that the static snapshot is being rendered and the UI thread hides the overlaid toolbar so that the static snapshot is visible. 4) The UI thread notifies the compositor thread that the UI toolbar has been hidden. This lets the C++ DynamicToolbarAnimator know it is allowed to animate static snapshot. 5) The C++ DynamicToolbarAnimator intercepts input events in APZCTreeManager::ReceiveInputEvent and moves the static snapshot offscreen while at the same time expanding the fixed frame margin to completely cover the remainder of the surface. 6) Once the static toolbar is hidden, The viewport is resized and events are processed as usual. Revealing the toolbar would work as follows: 1) The UI toolbar elements start hidden and the static snapshot scrolled off the screen. 2) The user swipes or drags down. 3) The C++ DynamicToolbarAnimator intercepts input events in APZCTreeManager::ReceiveInputEvent and moves the static snapshot onscreen while at the same time resizing the fixed frame margin so that all fixed elements will remain visible. 4) Once the static snapshot is fully visible, the static snapshot is pinned and the UI thread is notified to show the UI toolbar elements. The viewport is resized at this time. Event translation: Since the content will not always be rendered up at the top edge of the surface widget, events will need to be translated to accommodate for the content displacement. The C++ DynamicToolbarAnimator would apply this translation when it intercepts the event in APZCTreeManager::ReceiveInputEvent. Where the static snapshot is rendered: The C++ DynamicToolbarAnimator would have the additional task of adding a translation prior to rendering the content in LayerManagerComposite::Render if the static snapshot is visible. The static snapshot would also be blitted at this point. UI thread takes back control: The UI thread will be able to at any point request that the static toolbar be pinned visible. Once the UI thread has been informed that the static snapshot is visible, the UI toolbar elements may be shown again. The C++ DynamicToolbarAnimator must get permission from the UI thread before it may hide and/or animate the static snapshot again. Toolbar display flow control: When a page is loaded, the UI toolbar elements are displayed and the C++ DynamicToolbarAnimator is offsetting content so it is rendered starting below the bottom of the toolbar. The C++ DynamicToolbarAnimator is in a state that prevents it from animating the toolbar at this time. The C++ DynamicToolbarAnimator is intercepting events in APZCTreeManager::ReceiveInputEvent so they may be translated. Once content begins to scroll, the compositor thread must notify the UI thread that it wishes to animate the toolbar. The UI thread responds by sending a static snapshot of the UI toolbar to the compositor thread. This may be accomplished by either sending a Surface over the binder or copying the content of a Surface into shared memory and then shared via IPDL. Once the compositor thread has received the static snapshot and render it once, the compositor thread notifies the UI thread that it is now rendering the static snapshot underneath the UI toolbar elements. At this point the compositor thread is still in a state the prevents it from begin to animate the static snapshot even if the user is scrolling. It must wait for the UI thread to notify it back that the UI toolbar has been hidden. The UI thread may not immediately notify the compositor thread that it may begin to animate. The UI thread may want the toolbar to remain visible until certain conditions are met such as if the page load progress bar is still visible. Once the UI thread determines that the compositor thread may start animating the static snapshot, it hides the toolbar and notifies the compositor thread that it may start animating. The compositor thread may not immediately start animating the static snapshot if at the time it receives the notification there are no user events that would cause the static snapshot to animate. Fullscreen: When a page goes full screen, some coordination may be required to ensure a blank space is not visible when the UI toolbar is hidden. Copy/Cut/Paste on devices with a CCP Toolbar: On some Android devices (older versions of android, CyanogenMod, etc), the copy/cut/paste menu is a toolbar across the top of the page instead of a floating toolbar. Additional coordination might be need to ensure it displays properly. It is important that the content does not shift under the user’s finger when text is being selected. Sharing the FrameMetric with the UI thread: The UI thread will still need a copy of the current FrameMetric values. However, it will not need to be updated in real time. All UI elements that are positioned based on the current FrameMetric values are hidden during a scroll. Thus, the UI thread only needs the FrameMetric values when they are in a steady state. This means the compositor thread only needs to update when a scroll or fling completes. The UI thread would probably need to keep track of when the FrameMetric becomes stale and not display floating UI elements until the FrameMetric has been updated from the compositor thread. Partially visible static snapshot: If the user removes their finger before a drag has completely hidden or revealed the static snapshot, the C++ DynamicToolbarAnimator will need to finish either hiding or revealing the static snapshot. The heuristic will mirror what is currently done in the Java DynamicToolbarAnimator. It was suggested that the C++ DynamicToolbarAnimator would be able to use the Vsync notification to advance the animation once there are no longer user events to drive the translation of the static snapshot. GeckoView API: GeckoView consumers will be presented with a very simple API for dynamic toolbar support. The consumer registers a toolbar with GeckoView. GeckoView will be responsible for managing visibility of the toolbar. The consumer will be able to request the toolbar be made visible at any time and choose if it is visible instantly or animated. A callback maybe provide to notify the consumer when the toolbar is made visible or invisible. Scroll to end-of-page special case: When the user scrolls to the end of a page, the UI toolbar should be made visible again. 1) C++ DynamicToolbarAnimator is notified the end of page has been reached. 2) The static snapshot is animated into view. It is important that the bottom of the page remain at the bottom of the surface so that the content does not appear to shift off the screen. 3) Once the static snapshot is visible, the UI thread is notified to show the UI toolbar elements.

Comment on attachment 8836268 [details] [diff] [review] 0001-Convert-UiCompositorController-from-singleton-to-one-per-CompositorSession-17021015-e245404.patch For the dynamic toolbar rework, I need to be able to communicate from the compositor thread back to the UI thread. It is easier to do this if each CompositorSession has a UiCompositorControllerChild instance instead of a singleton. Is this the correct approach? Do you see any issues that may have been introduced? Sorry, git diff made a mess of this patch.