Found while fuzzing m-c 20210906-aca153106940 (--enable-debug --enable-fuzzing)

Assertion failure: false (Two layers that scroll together have different ancestor transforms), at src/gfx/layers/apz/src/APZCTreeManager.cpp:1409

#0 0x7f16acf67680 in mozilla::layers::HitTestingTreeNode* mozilla::layers::APZCTreeManager::PrepareNodeForLayer<mozilla::layers::WebRenderScrollDataWrapper>(mozilla::RecursiveMutexAutoLock const&, mozilla::layers::WebRenderScrollDataWrapper const&, mozilla::layers::FrameMetrics const&, mozilla::layers::LayersId, mozilla::Maybe<mozilla::layers::ZoomConstraints> const&, mozilla::layers::AncestorTransform const&, mozilla::layers::HitTestingTreeNode*, mozilla::layers::HitTestingTreeNode*, mozilla::layers::APZCTreeManager::TreeBuildingState&) src/gfx/layers/apz/src/APZCTreeManager.cpp:1407:9
#1 0x7f16acf6367e in void mozilla::layers::APZCTreeManager::UpdateHitTestingTreeImpl<mozilla::layers::WebRenderScrollDataWrapper>(mozilla::layers::WebRenderScrollDataWrapper const&, bool, mozilla::layers::LayersId, unsigned int)::'lambda'(mozilla::layers::WebRenderScrollDataWrapper)::operator()(mozilla::layers::WebRenderScrollDataWrapper) const src/gfx/layers/apz/src/APZCTreeManager.cpp:479:38
#2 0x7f16acf62e42 in _ZN7mozilla6layersL11ForEachNodeINS0_15ReverseIteratorENS0_26WebRenderScrollDataWrapperEZNS0_15APZCTreeManager24UpdateHitTestingTreeImplIS3_EEvRKT_bNS0_8LayersIdEjEUlS3_E_ZNS5_IS3_EEvS8_bS9_jEUlS3_E0_EENSt9enable_ifIXaasr3stdE9is_same_vIDTclfp0_fp_EEvEsr3stdE9is_same_vIDTclfp1_fp_EEvEEvE4typeET0_RKT1_RKT2_ src/gfx/layers/TreeTraversal.h:139:3
#3 0x7f16acf62ec7 in _ZN7mozilla6layersL11ForEachNodeINS0_15ReverseIteratorENS0_26WebRenderScrollDataWrapperEZNS0_15APZCTreeManager24UpdateHitTestingTreeImplIS3_EEvRKT_bNS0_8LayersIdEjEUlS3_E_ZNS5_IS3_EEvS8_bS9_jEUlS3_E0_EENSt9enable_ifIXaasr3stdE9is_same_vIDTclfp0_fp_EEvEsr3stdE9is_same_vIDTclfp1_fp_EEvEEvE4typeET0_RKT1_RKT2_ src/gfx/layers/TreeTraversal.h:143:5
#4 0x7f16acf62ec7 in _ZN7mozilla6layersL11ForEachNodeINS0_15ReverseIteratorENS0_26WebRenderScrollDataWrapperEZNS0_15APZCTreeManager24UpdateHitTestingTreeImplIS3_EEvRKT_bNS0_8LayersIdEjEUlS3_E_ZNS5_IS3_EEvS8_bS9_jEUlS3_E0_EENSt9enable_ifIXaasr3stdE9is_same_vIDTclfp0_fp_EEvEsr3stdE9is_same_vIDTclfp1_fp_EEvEEvE4typeET0_RKT1_RKT2_ src/gfx/layers/TreeTraversal.h:143:5
#5 0x7f16acf62ec7 in _ZN7mozilla6layersL11ForEachNodeINS0_15ReverseIteratorENS0_26WebRenderScrollDataWrapperEZNS0_15APZCTreeManager24UpdateHitTestingTreeImplIS3_EEvRKT_bNS0_8LayersIdEjEUlS3_E_ZNS5_IS3_EEvS8_bS9_jEUlS3_E0_EENSt9enable_ifIXaasr3stdE9is_same_vIDTclfp0_fp_EEvEsr3stdE9is_same_vIDTclfp1_fp_EEvEEvE4typeET0_RKT1_RKT2_ src/gfx/layers/TreeTraversal.h:143:5
#6 0x7f16acf62ec7 in _ZN7mozilla6layersL11ForEachNodeINS0_15ReverseIteratorENS0_26WebRenderScrollDataWrapperEZNS0_15APZCTreeManager24UpdateHitTestingTreeImplIS3_EEvRKT_bNS0_8LayersIdEjEUlS3_E_ZNS5_IS3_EEvS8_bS9_jEUlS3_E0_EENSt9enable_ifIXaasr3stdE9is_same_vIDTclfp0_fp_EEvEsr3stdE9is_same_vIDTclfp1_fp_EEvEEvE4typeET0_RKT1_RKT2_ src/gfx/layers/TreeTraversal.h:143:5
#7 0x7f16acefa1bb in void mozilla::layers::APZCTreeManager::UpdateHitTestingTreeImpl<mozilla::layers::WebRenderScrollDataWrapper>(mozilla::layers::WebRenderScrollDataWrapper const&, bool, mozilla::layers::LayersId, unsigned int) src/gfx/layers/apz/src/APZCTreeManager.cpp:438:5
#8 0x7f16acf71bda in UpdateHitTestingTree src/gfx/layers/apz/src/APZCTreeManager.cpp:720:3
#9 0x7f16acf71bda in operator() src/gfx/layers/apz/src/APZUpdater.cpp:204:25
#10 0x7f16acf71bda in mozilla::detail::RunnableFunction<mozilla::layers::APZUpdater::UpdateScrollDataAndTreeState(mozilla::layers::LayersId, mozilla::layers::LayersId, mozilla::wr::Epoch const&, mozilla::layers::WebRenderScrollData&&)::$_31>::Run() /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:531:5
#11 0x7f16acf227df in mozilla::layers::APZUpdater::ProcessQueue() src/gfx/layers/apz/src/APZUpdater.cpp:466:23
#12 0x7f16acf2204a in mozilla::layers::APZUpdater::CompleteSceneSwap(mozilla::wr::WrWindowId const&, mozilla::wr::WrPipelineInfo const&) src/gfx/layers/apz/src/APZUpdater.cpp:120:12
#13 0x7f16acf25519 in apz_post_scene_swap src/gfx/layers/apz/src/APZUpdater.cpp:534:3
#14 0x7f16b3e5470d in _$LT$webrender_bindings..bindings..APZCallbacks$u20$as$u20$webrender..renderer..SceneBuilderHooks$GT$::post_scene_swap::h6285849f0cf19fdc src/gfx/webrender_bindings/src/bindings.rs:976:13
#15 0x7f16b41450d9 in webrender::scene_builder_thread::SceneBuilderThread::forward_built_transactions::h6c07854e97bd6a1c src/gfx/wr/webrender/src/scene_builder_thread.rs:699:13
#16 0x7f16b41450d9 in webrender::scene_builder_thread::SceneBuilderThread::run::h5813092075594cf2 src/gfx/wr/webrender/src/scene_builder_thread.rs:313:21
#17 0x7f16b3ebd6e9 in webrender::renderer::Renderer::new::_$u7b$$u7b$closure$u7d$$u7d$::h8489d50038ddbb44 src/gfx/wr/webrender/src/renderer/mod.rs:1230:13
#18 0x7f16b3ebd6e9 in std::sys_common::backtrace::__rust_begin_short_backtrace::hc976d6d50b6d9ed4 /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/sys_common/backtrace.rs:125:18
#19 0x7f16b3edae0e in std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h1f8f7716cd7d4482 /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/thread/mod.rs:481:17
#20 0x7f16b3edae0e in _$LT$std..panic..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h09819f00e606cdc7 /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panic.rs:347:9
#21 0x7f16b3edae0e in std::panicking::try::do_call::h9b682a12419da40e /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panicking.rs:401:40
#22 0x7f16b3edae0e in std::panicking::try::h32cd152da2f6a3ae /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panicking.rs:365:19
#23 0x7f16b3edae0e in std::panic::catch_unwind::h5849d47c1e6cda14 /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panic.rs:434:14
#24 0x7f16b3edae0e in std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h68a449b9d6d1d519 /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/thread/mod.rs:480:30
#25 0x7f16b3edae0e in core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hc86aa4554ea1c4fa /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/ops/function.rs:227:5
#26 0x7f16b54ae736 in _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h7ece6cfefaff1005 /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/alloc/src/boxed.rs:1575:9
#27 0x7f16b54ae736 in _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::hb8b48e55c21f193e /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/alloc/src/boxed.rs:1575:9
#28 0x7f16b54ae736 in std::sys::unix::thread::Thread::new::thread_start::h8c7c4450dba62914 /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/sys/unix/thread.rs:71:17
#29 0x7f16c1570608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477:8
#30 0x7f16c1138292 in clone /build/glibc-eX1tMB/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

A Pernosco session is available here: https://pernos.co/debug/yblpQ1xWOIVneHcbNM5d0w/index.html

Bugmon Analysis
Verified bug as reproducible on mozilla-central 20210908032417-a4d2ca53b2a4.
Failed to bisect testcase (Testcase reproduces on start build!):

Start: 6cc2266faca2a2301c81f3442d6d893ad3fc9fdf (20200909093957)
End: aca153106940b800cac068609f552fc82b2ba2d1 (20210906214243)
BuildFlags: BuildFlags(asan=False, tsan=False, debug=True, fuzzing=True, coverage=False, valgrind=False, no_opt=False, fuzzilli=False)

WRLSD tree:

WebRenderLayerScrollData(0x7f8ecea74008), descendantCount=6, visible=[]
    WebRenderLayerScrollData(0x7f8ecea74198), descendantCount=5, visible=[]
        WebRenderLayerScrollData(0x7f8ecea74968), descendantCount=0, metadata0={ [metrics={ [cb=(x=0, y=0, w=1280, h=887)] [sr=(x=0, y=0, w=1280, h=887)] [s=(0,0)] [dp=(x=0, y=0, w=1280, h=887)] [cdp=(x=0, y=0, w=0, h=0)] [rcs=(1280 x 887)] [v=(x=0, y=0, w=1280, h=887)] [z=(ld=1.000 r=1.000 cr=1 z=1 er=1 )] [u=(0 1)] scrollId=2 [rcd] }] [color=dev_rgba(255, 255, 255, 1.000000)] [overscroll=auto] [1 scrollupdates] }, visible=[]
        WebRenderLayerScrollData(0x7f8ecea74328), descendantCount=3, visible=[], fixedContainer=2, fixedAnimation=0x0, sideBits=0x0
            WebRenderLayerScrollData(0x7f8ecea744b8), descendantCount=2, ancestorTransform=[ 1 0; 0 1; 8 16; ] (asr=0), visible=[]
                WebRenderLayerScrollData(0x7f8ecea74648), descendantCount=1, metadata0={ [metrics={ [cb=(x=0, y=0, w=1280, h=887)] [sr=(x=0, y=0, w=1280, h=887)] [s=(0,0)] [dp=(x=0, y=0, w=1280, h=887)] [cdp=(x=0, y=0, w=0, h=0)] [rcs=(1280 x 887)] [v=(x=0, y=0, w=1280, h=887)] [z=(ld=1.000 r=1.000 cr=1 z=1 er=1 )] [u=(0 1)] scrollId=2 [rcd] }] [color=dev_rgba(255, 255, 255, 1.000000)] [overscroll=auto] [1 scrollupdates] }, visible=[]
                    WebRenderLayerScrollData(0x7f8ecea747d8), descendantCount=0, visible=[]

There are two nodes with scrollId=2, and there is an ancestorTransform=[ 1 0; 0 1; 8 16; ] present only on the path from the second node to their common ancestor.

scrollId=2 is the root, and the second occurrence of it is inside a fixed subtree, which seems unexpected.

Here's a Gecko display list:

SolidColor p=0x7f8ecea6c3a8 f=0x7f8ecea1f020(Viewport(-1)) key=53 bounds(0,0,76800,53220) layerBounds(0,0,76800,53220) visible(0,0,76800,53220) building(0,0,76800,53220) componentAlpha(0,0,0,0) clip() asr() clipChain() uniform ref=0x7f8ecea1f020 agr=0x7f8ecea1f020 (opaque 0,0,76800,53220) (rgba 255,255,255,255)
CompositorHitTestInfo p=0x7f8ecea6b020 f=0x7f8ecea1f1a8(HTMLScroll(html)(-1)) key=25 bounds(0,0,0,0) layerBounds(0,0,0,0) visible(0,0,76800,53220) building(0,0,76800,53220) componentAlpha(0,0,0,0) clip() asr() clipChain() ref=0x7f8ecea1f020 agr=0x7f8ecea1f020 hitTestInfo(0x1) hitTestArea(0,0,76800,53220)
AsyncZoom p=0x7f8ecea6c210 f=0x7f8ecea1f1a8(HTMLScroll(html)(-1)) key=2 bounds(0,0,76800,53220) layerBounds(0,0,76800,53220) visible(0,0,76800,53220) building(0,0,76800,53220) componentAlpha(0,0,0,0) clip(0,0,76800,53220) asr() clipChain(0x7f8ecea6b1d0 <0,0,76800,53220> [root asr]) ref=0x7f8ecea1f020 agr=0x7f8ecea1f020 (opaque 0,0,76800,53220) (flags 0x0) (scrolltarget 0)
  CanvasBackgroundColor p=0x7f8ecea6b218 f=0x7f8ecea1f0c8(Canvas(html)(-1)) key=16 bounds(0,0,76800,53220) layerBounds(0,0,76800,53220) visible(0,0,76800,53220) building(0,0,76800,53220) componentAlpha(0,0,0,0) clip(0,0,76800,53220) asr(<0x7f8ecea1f248>) clipChain(0x7f8ecea6b2f0 <0,0,76800,53220> [0x7f8ecea1f248], 0x7f8ecea6b1d0 <0,0,76800,53220> [root asr]) uniform ref=0x7f8ecea1f020 agr=0x7f8ecea1f0c8 (opaque 0,0,76800,53220) (rgba 255,255,255,255)
  CompositorHitTestInfo p=0x7f8ecea6b0f8 f=0x7f8ecea1f0c8(Canvas(html)(-1)) key=281 bounds(0,0,0,0) layerBounds(0,0,0,0) visible(0,0,76800,53220) building(0,0,76800,53220) componentAlpha(0,0,0,0) clip() asr(<0x7f8ecea1f248>) clipChain(0x7f8ecea6b1d0 <0,0,76800,53220> [root asr]) ref=0x7f8ecea1f020 agr=0x7f8ecea1f0c8 hitTestInfo(0x1) hitTestArea(0,0,76800,53220)
  Mask p=0x7f8ecea6bfb0 f=0x7f8ecea1fae0(Block(html)(-1)) key=42 bounds(480,960,0,0) layerBounds(480,960,0,0) visible(0,0,76800,53220) building(0,0,76800,53220) componentAlpha(0,0,0,0) clip() asr(<0x7f8ecea1f248>) clipChain(0x7f8ecea6b1d0 <0,0,76800,53220> [root asr]) ref=0x7f8ecea1f020 agr=0x7f8ecea1f0c8 effects=(clip(basic-shape))
    CompositorHitTestInfo p=0x7f8ecea6b338 f=0x7f8ecea1fae0(Block(html)(-1)) key=25 bounds(0,0,0,0) layerBounds(0,0,0,0) visible(0,0,60,60) building(0,0,60,60) componentAlpha(0,0,0,0) clip(0,0,60,60) asr(<0x7f8ecea1f248>) clipChain(0x7f8ecea6b410 <0,0,60,60> [0x7f8ecea1f248], 0x7f8ecea6b1d0 <0,0,76800,53220> [root asr]) ref=0x7f8ecea1f020 agr=0x7f8ecea1f0c8 hitTestInfo(0x3) hitTestArea(0,0,76800,2100)
    Mask p=0x7f8ecea6bd50 f=0x7f8ecea1fba8(Block(body)(1)) key=42 bounds(480,960,0,0) layerBounds(480,960,0,0) visible(0,0,600,1080) building(0,0,600,1080) componentAlpha(0,0,0,0) clip() asr(<0x7f8ecea1f248>) clipChain(0x7f8ecea6b1d0 <0,0,76800,53220> [root asr]) ref=0x7f8ecea1f020 agr=0x7f8ecea1f0c8 effects=(clip(basic-shape))
      CompositorHitTestInfo p=0x7f8ecea6b458 f=0x7f8ecea1fba8(Block(body)(1)) key=25 bounds(0,0,0,0) layerBounds(0,0,0,0) visible(480,420,0,0) building(480,420,0,0) componentAlpha(0,0,0,0) clip(480,420,0,0) asr(<0x7f8ecea1f248>) clipChain(0x7f8ecea6b530 <480,420,0,0> [0x7f8ecea1f248], 0x7f8ecea6b1d0 <0,0,76800,53220> [root asr]) ref=0x7f8ecea1f020 agr=0x7f8ecea1f0c8 hitTestInfo(0x3) hitTestArea(480,480,75840,1140)
      FixedPosition p=0x7f8ecea6bb60 f=0x7f8ecea1fcc0(Block(hr)(0) class:a) key=31 bounds(480,960,0,0) layerBounds(0,0,0,0) visible(480,960,120,120) building(480,960,120,120) componentAlpha(0,0,0,0) clip(0,0,76800,53220) asr() clipChain(0x7f8ecea6b530 <480,420,0,0> [0x7f8ecea1f248], 0x7f8ecea6b1d0 <0,0,76800,53220> [root asr]) ref=0x7f8ecea1f020 agr=0x7f8ecea1fcc0 (containerASR <0x7f8ecea1f248>) (scrolltarget 2)
        nsDisplayTransform p=0x7f8ecea6b970 f=0x7f8ecea1fcc0(Block(hr)(0) class:a) key=73 bounds(480,960,0,0) layerBounds(0,0,0,0) visible(480,960,120,120) building(480,960,120,120) componentAlpha(0,0,0,0) clip() asr() clipChain() ref=0x7f8ecea1f020 agr=0x7f8ecea1fcc0[ 1 0; 0 1; 8 16; ] prerender(no) childrenBuildingRect(x=0, y=0, w=120, h=120)
          Opacity p=0x7f8ecea6b810 f=0x7f8ecea1fcc0(Block(hr)(0) class:a) key=43 bounds(0,0,0,0) layerBounds(0,0,0,0) visible(0,0,120,120) building(0,0,120,120) componentAlpha(0,0,0,0) clip() asr(<0x7f8ecea1f248>) clipChain() ref=0x7f8ecea1fcc0 agr=0x7f8ecea1fcc0 (opacity 0, mChildOpacityState: Unknown)
            Mask p=0x7f8ecea6b698 f=0x7f8ecea1fcc0(Block(hr)(0) class:a) key=42 bounds(0,0,0,0) layerBounds(0,0,0,0) visible(0,0,120,120) building(0,0,120,120) componentAlpha(0,0,0,0) clip() asr() clipChain() ref=0x7f8ecea1fcc0 agr=0x7f8ecea1fcc0 effects=(clip(basic-shape))
              CompositorHitTestInfo p=0x7f8ecea6b578 f=0x7f8ecea1fcc0(Block(hr)(0) class:a) key=25 bounds(0,0,0,0) layerBounds(0,0,0,0) visible(0,0,60,60) building(0,0,60,60) componentAlpha(0,0,0,0) clip(0,-60,60,120) asr() clipChain(0x7f8ecea6b650 <0,-60,60,120> [root asr]) ref=0x7f8ecea1fcc0 agr=0x7f8ecea1fcc0 hitTestInfo(0x3) hitTestArea(0,0,120,120)

The only item in the FixedPosition's subtree which has a non-null ASR is the Opacity item. The next step is to understand why it gets the root scroll frame's ASR.

Is it this code?

https://searchfox.org/mozilla-central/rev/1761c710b035d7dc8892dfa8e56589b4e095221f/layout/generic/nsIFrame.cpp#3559

(clip-path plus fixed pos causes a lot of the ASR asserts that fuzzers run into, so perhaps in this case we don't hit those ASR asserts but it causes a problem later on in the painting pipeline)

Depends on D125495

Scrolling over the fixed element now scrolls the root scroll frame,
not the subframe.

This aligns with both Chrome's behaviour on this testcase, and
existing practice in our code for scroll handoff to generally
follow the frame tree.

Depends on D125496

Depends on D125497

This might fix bug 1649668, bug 1427792, bug 1654315, and bug 1729589.

(In reply to Timothy Nikkel (:tnikkel) from comment #11)

This might fix bug 1649668, bug 1427792, bug 1654315, and bug 1729589.

Good catch. I went through the testcases of these bugs and found that:

• The patch here does fix bug 1649668 and bug 1654315.
• The patch here does not fix bug 1729589.
• For bug 1427792, I can't reproduce it so I'm not sure.

(In reply to Botond Ballo [:botond] from comment #12)

• For bug 1427792, I can't reproduce it so I'm not sure.

After trying again with provided prefs.js, I was able to reproduce bug 1427792, and confirmed that the patch here does not fix it.

https://hg.mozilla.org/mozilla-central/rev/935ed89e9c3d
https://hg.mozilla.org/mozilla-central/rev/5196788052a1
https://hg.mozilla.org/mozilla-central/rev/5d5c2ceea4f3
https://hg.mozilla.org/mozilla-central/rev/8dc2af40602c

Bugmon Analysis
Verified bug as fixed on rev mozilla-central 20210918094657-b01f4c5fd7b7.
Removing bugmon keyword as no further action possible. Please review the bug and re-add the keyword for further analysis.

:botond, since this bug contains a bisection range, could you fill (if possible) the regressed_by field?
For more information, please visit auto_nag documentation.

Sorry, wrong needinfo because of a bug in the bot.