Just randomly happens if I scroll a form inside a page back up.

Assertion failure: result.mScrollId == ScrollableLayerGuid::NULL_SCROLL_ID, at /mnt/code/firefox/gfx/layers/apz/src/APZCTreeManager.cpp:2834
#01: mozilla::layers::APZCTreeManager::GetTargetAPZC(mozilla::gfx::PointTyped<mozilla::ScreenPixel, float> const&) (/mnt/code/firefox/gfx/layers/apz/src/APZCTreeManager.cpp:2785)
#02: mozilla::layers::APZCTreeManager::ReceiveInputEvent(mozilla::InputData&) (/mnt/code/firefox/gfx/layers/apz/src/APZCTreeManager.cpp:1511)
#03: {virtual override thunk({offset(-16)}, mozilla::layers::APZCTreeManager::ReceiveInputEvent(mozilla::InputData&))} (/mnt/code/firefox/gfx/layers/apz/src/APZCTreeManager.cpp:0)
#04: mozilla::layers::APZInputBridgeParent::RecvReceiveMouseInputEvent(mozilla::MouseInput const&, mozilla::layers::APZEventResult*, mozilla::MouseInput*) (/mnt/code/firefox/gfx/layers/ipc/APZInputBridgeParent.cpp:42)
#05: mozilla::layers::PAPZInputBridgeParent::OnMessageReceived(IPC::Message const&, IPC::Message*&) (/mnt/code/firefox/obj-x86_64-pc-linux-gnu/ipc/ipdl/PAPZInputBridgeParent.cpp:246)
#06: mozilla::gfx::PGPUParent::OnMessageReceived(IPC::Message const&, IPC::Message*&) (/mnt/code/firefox/obj-x86_64-pc-linux-gnu/ipc/ipdl/PGPUParent.cpp:1552)
#07: mozilla::ipc::MessageChannel::DispatchSyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&, IPC::Message*&) (/mnt/code/firefox/ipc/glue/MessageChannel.cpp:2155)
#08: mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) (/mnt/code/firefox/obj-x86_64-pc-linux-gnu/dist/include/mozilla/UniquePtrExtensions.h:139)
#09: mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&) (/mnt/code/firefox/ipc/glue/MessageChannel.cpp:0)
#10: mozilla::ipc::MessageChannel::MessageTask::Run() (/mnt/code/firefox/ipc/glue/MessageChannel.cpp:1991)
#11: nsThread::ProcessNextEvent(bool, bool*) (/mnt/code/firefox/xpcom/threads/nsThread.cpp:1200)
#12: NS_ProcessNextEvent(nsIThread*, bool) (/mnt/code/firefox/xpcom/threads/nsThreadUtils.cpp:481)
#13: mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) (/mnt/code/firefox/ipc/glue/MessagePump.cpp:87)
#14: MessageLoop::RunInternal() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:315)
#15: MessageLoop::Run() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:291)
#16: nsBaseAppShell::Run() (/mnt/code/firefox/widget/nsBaseAppShell.cpp:139)
#17: XRE_RunAppShell() (/mnt/code/firefox/toolkit/xre/nsEmbedFunctions.cpp:909)
#18: mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) (/mnt/code/firefox/ipc/glue/MessagePump.cpp:237)
#19: MessageLoop::RunInternal() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:315)
#20: MessageLoop::Run() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:291)
#21: XRE_InitChildProcess(int, char**, XREChildData const*) (/mnt/code/firefox/toolkit/xre/nsEmbedFunctions.cpp:740)
#22: content_process_main(mozilla::Bootstrap*, int, char**) (/mnt/code/firefox/ipc/contentproc/plugin-container.cpp:57)
#23: main (/mnt/code/firefox/browser/app/nsBrowserApp.cpp:303)
#24: __libc_start_main (/usr/lib/haswell/libc.so.6 + 0x270d3)
#25: _start (/mnt/code/firefox/obj-x86_64-pc-linux-gnu/dist/bin/firefox + 0x3f02e)
#26: ??? (???:???)

Program /mnt/code/firefox/obj-x86_64-pc-linux-gnu/dist/bin/firefox (pid = 158781) received signal 11.
Stack:

###!!! [Parent][MessageChannel::SendAndWait] Error: Channel error: cannot send/recv

#01: mozilla::layers::APZCTreeManager::GetAPZCAtPointWR(mozilla::gfx::PointTyped<mozilla::ScreenPixel, float> const&, mozilla::RecursiveMutexAutoLock const&) (/mnt/code/firefox/gfx/layers/apz/src/APZCTreeManager.cpp:2834)
#02: mozilla::layers::APZCTreeManager::GetTargetAPZC(mozilla::gfx::PointTyped<mozilla::ScreenPixel, float> const&) (/mnt/code/firefox/gfx/layers/apz/src/APZCTreeManager.cpp:2785)
#03: mozilla::layers::APZCTreeManager::ReceiveInputEvent(mozilla::InputData&) (/mnt/code/firefox/gfx/layers/apz/src/APZCTreeManager.cpp:1511)
#04: {virtual override thunk({offset(-16)}, mozilla::layers::APZCTreeManager::ReceiveInputEvent(mozilla::InputData&))} (/mnt/code/firefox/gfx/layers/apz/src/APZCTreeManager.cpp:0)
#05: mozilla::layers::APZInputBridgeParent::RecvReceiveMouseInputEvent(mozilla::MouseInput const&, mozilla::layers::APZEventResult*, mozilla::MouseInput*) (/mnt/code/firefox/gfx/layers/ipc/APZInputBridgeParent.cpp:42)
#06: mozilla::layers::PAPZInputBridgeParent::OnMessageReceived(IPC::Message const&, IPC::Message*&) (/mnt/code/firefox/obj-x86_64-pc-linux-gnu/ipc/ipdl/PAPZInputBridgeParent.cpp:246)
#07: mozilla::gfx::PGPUParent::OnMessageReceived(IPC::Message const&, IPC::Message*&) (/mnt/code/firefox/obj-x86_64-pc-linux-gnu/ipc/ipdl/PGPUParent.cpp:1552)
#08: mozilla::ipc::MessageChannel::DispatchSyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&, IPC::Message*&) (/mnt/code/firefox/ipc/glue/MessageChannel.cpp:2155)
#09: mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) (/mnt/code/firefox/obj-x86_64-pc-linux-gnu/dist/include/mozilla/UniquePtrExtensions.h:139)
#10: mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&) (/mnt/code/firefox/ipc/glue/MessageChannel.cpp:0)
#11: mozilla::ipc::MessageChannel::MessageTask::Run() (/mnt/code/firefox/ipc/glue/MessageChannel.cpp:1991)
#12: nsThread::ProcessNextEvent(bool, bool*) (/mnt/code/firefox/xpcom/threads/nsThread.cpp:1200)
#13: NS_ProcessNextEvent(nsIThread*, bool) (/mnt/code/firefox/xpcom/threads/nsThreadUtils.cpp:481)
#14: mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) (/mnt/code/firefox/ipc/glue/MessagePump.cpp:87)
#15: MessageLoop::RunInternal() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:315)
#16: MessageLoop::Run() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:291)
#17: nsBaseAppShell::Run() (/mnt/code/firefox/widget/nsBaseAppShell.cpp:139)
#18: XRE_RunAppShell() (/mnt/code/firefox/toolkit/xre/nsEmbedFunctions.cpp:909)
#19: mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) (/mnt/code/firefox/ipc/glue/MessagePump.cpp:237)
#20: MessageLoop::RunInternal() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:315)
#21: MessageLoop::Run() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:291)
#22: XRE_InitChildProcess(int, char**, XREChildData const*) (/mnt/code/firefox/toolkit/xre/nsEmbedFunctions.cpp:740)
#23: content_process_main(mozilla::Bootstrap*, int, char**) (/mnt/code/firefox/ipc/contentproc/plugin-container.cpp:57)
#24: main (/mnt/code/firefox/browser/app/nsBrowserApp.cpp:303)
#25: __libc_start_main (/usr/lib/haswell/libc.so.6 + 0x270d3)
#26: _start (/mnt/cod

I replied on Matrix, but for posterity:

• apply this patch
• turn on the gfx.webrender.dl.dump-content pref, which will dump the gecko and WR display lists
• run with MOZ_LOG="apz.manager:4,apz.controller:4,apz.inputqueue:4

This will spew out a bunch of stuff to stderr, hopefully this will give me enough information to figure out what the hit-test is hitting and why APZ doesn't have the corresponding node for that.

Attaching the console output with the environment and pref, as requested.

Tested with the patch in https://github.com/staktrace/gecko/commit/f061b359d341b7e742bd280f1a06d6dbbf5e1003
Attached the log output (65Mb...) compressed as ZIP. Here is the updated call stack:

Program /mnt/code/firefox/obj-x86_64-pc-linux-gnu/dist/bin/firefox (pid = 168924) received signal 11.
Stack:
#01: mozilla::wr::RenderThread::UpdateAndRender(mozilla::wr::WrWindowId, mozilla::layers::BaseTransactionId<mozilla::VsyncIdType> const&, mozilla::TimeStamp const&, bool, mozilla::Maybe<mozilla::gfx::IntSizeTyped<mozilla::gfx::UnknownUnits> > const&, mozilla::Maybe<mozilla::wr::ImageFormat> const&, mozilla::Maybe<mozilla::Range<unsigned char> > const&) (/mnt/code/firefox/gfx/webrender_bindings/RenderThread.cpp:535)
#02: mozilla::wr::RenderThread::HandleFrameOneDoc(mozilla::wr::WrWindowId, bool) (/mnt/code/firefox/gfx/webrender_bindings/RenderThread.cpp:363)
#03: mozilla::detail::RunnableMethodImpl<mozilla::wr::RenderThread*, void (mozilla::wr::RenderThread::*)(mozilla::wr::WrWindowId, bool), true, (mozilla::RunnableKind)0, mozilla::wr::WrWindowId, bool>::Run() (/mnt/code/firefox/obj-x86_64-pc-linux-gnu/dist/include/nsThreadUtils.h:1223)
#04: MessageLoop::RunTask(already_AddRefed<nsIRunnable>) (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:443)
#05: MessageLoop::DeferOrRunPendingTask(MessageLoop::PendingTask&&) (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:450)
#06: MessageLoop::DoWork() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:523)
#07: base::MessagePumpDefault::Run(base::MessagePump::Delegate*) (/mnt/code/firefox/ipc/chromium/src/base/message_pump_default.cc:35)
#08: MessageLoop::RunInternal() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:315)
#09: MessageLoop::Run() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:291)
#10: base::Thread::ThreadMain() (/mnt/code/firefox/ipc/chromium/src/base/thread.cc:195)
#11: ThreadFunc(void*) (/mnt/code/firefox/ipc/chromium/src/base/platform_thread_posix.cc:41)
#12: ??? (/usr/lib/libpthread.so.0 + 0x9606)
#13: clone (/usr/lib/haswell/libc.so.6 + 0x120753)
#14: ??? (???:???)
Sleeping for 300 seconds.
Type 'gdb /mnt/code/firefox/obj-x86_64-pc-linux-gnu/dist/bin/firefox 168924' to attach your debugger to this thread.

Program /mnt/code/firefox/obj-x86_64-pc-linux-gnu/dist/bin/firefox (pid = 168924) received signal 11.
Stack:
#01: mozilla::layers::ContentCompositorBridgeParent::AllocPWebRenderBridgeParent(mozilla::wr::PipelineId const&, mozilla::gfx::IntSizeTyped<mozilla::LayoutDevicePixel> const&) (/mnt/code/firefox/gfx/layers/ipc/ContentCompositorBridgeParent.cpp:222)
#02: mozilla::layers::PCompositorBridgeParent::OnMessageReceived(IPC::Message const&) (/mnt/code/firefox/obj-x86_64-pc-linux-gnu/ipc/ipdl/PCompositorBridgeParent.cpp:1695)
#03: mozilla::layers::PCompositorManagerParent::OnMessageReceived(IPC::Message const&) (/mnt/code/firefox/obj-x86_64-pc-linux-gnu/ipc/ipdl/PCompositorManagerParent.cpp:197)
#04: mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&) (/mnt/code/firefox/ipc/glue/MessageChannel.cpp:2187)
#05: mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) (/mnt/code/firefox/ipc/glue/MessageChannel.cpp:2113)
#06: mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&) (/mnt/code/firefox/ipc/glue/MessageChannel.cpp:0)
#07: mozilla::ipc::MessageChannel::MessageTask::Run() (/mnt/code/firefox/ipc/glue/MessageChannel.cpp:1991)
#08: MessageLoop::RunTask(already_AddRefed<nsIRunnable>) (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:443)
#09: MessageLoop::DeferOrRunPendingTask(MessageLoop::PendingTask&&) (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:450)
#10: MessageLoop::DoWork() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:523)
#11: base::MessagePumpDefault::Run(base::MessagePump::Delegate*) (/mnt/code/firefox/ipc/chromium/src/base/message_pump_default.cc:35)
#12: MessageLoop::RunInternal() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:315)
#13: MessageLoop::Run() (/mnt/code/firefox/ipc/chromium/src/base/message_loop.cc:291)
#14: base::Thread::ThreadMain() (/mnt/code/firefox/ipc/chromium/src/base/thread.cc:195)
#15: ThreadFunc(void*) (/mnt/code/firefox/ipc/chromium/src/base/platform_thread_posix.cc:41)
#16: ??? (/usr/lib/libpthread.so.0 + 0x9606)
#17: clone (/usr/lib/haswell/libc.so.6 + 0x120753)
#18: ??? (???:???)
Sleeping for 300 seconds.

Repro steps used:

1. Open any post on "hacks.mozilla.org"
2. Scroll to the bottom, where the comment area is
3. Fill out the area with a ton of text, scroll it back and forth

Comment 5 seems like a different crash. It's certainly not the assertion failure from comment 0. But I'll try using the STR and reproducing locally on Monday.

Might be a recent-ish regression. I tried to reproduce in my local build from about a week ago and failed, and then I was able to easily reproduce after updating my build today.

Can repro, will investigate. Seems like there's a scrolltarget in the display list for which APZ doesn't have information even though I think it probably should. Not sure exactly why yet.

I looked at this a bit more. What seems to be happening is that the scrollframe inside the textarea has mWillBuildScrollableLayer set to false, and so never creates an ASR in the display list. Since there's no ASR, the code that we use to generate the WebRenderScrollData doesn't know about this scrollframe, and doesn't provide APZ with any metrics for it. However, the scrollframe does end up with scrollbars, and the scrollbar data references the scrollId of the scrollframe. So when the user tries to drag the scrollbar, the WR hit-test hits the scrollbar and returns that scrollid to APZ, but APZ has no metrics for it, and therefore fails the assertion.

It seems like the inactive-scrollframe hit-test info item here should be informing APZ of the existence of this scrollframe, but I guess it's not. I need to investigate that further.

Further investigation reveals that normally for inactive scrollframes (where mWillBuildScrollableLayer is false) the scrollbar data uses a NULL_SCROLL_ID scroll id. However this is not the case if IsMaybeScrollingActive() returns true here. In that case, the scrollbar data does use the actual scroll id. This might arise if, for example, the scrollframe has a will-change: scroll-position set but is actually inactive (perhaps due to falling outside the will-change budget).

I was able to reproduce this scenario and capture it in a mochitest. Now I need to figure out what the right fix is.

I wrote a fix that seems to work. Since the scrollbardata has the scroll id, we need to send metrics corresponding to that scroll id to APZ, which for an inactive scrollframe can be done by emitting a nsDisplayScrollInfo item.

We do this here but only when the item needs "hoisting" because it's inside an inactive layer manager. For WR we need to do it all the time that IsMaybeScrollingActive() is true. We don't need to worry about interaction with hoisted scrollinfo item because if we're inside an inactive layer manager then the non-hoisted one will get discarded anyway, and if we're not in an inactive layer manager then we won't emit the hoisted one. So we should never end up with duplicate scrollinfo items (although it wouldn't be a correctness problem even if we did).

Once that's in place the mochitest I wrote no longer asserts, but it does give different scrollid results for WR and non-WR, because in the non-WR codepath the hit-test finds the scrollinfo item and returns the scrollid from that, whereas with WR the hit-test info item is generated here and uses the scrollid of the enclosing ASR which is different. So I tweaked that too. With that the test passes and produces the same results with both WR and non-WR, and no assertion failure.

https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=44f8f3768ffd7f7ded616b5f42a81ca92f5eec2d

The user-observed behaviour should be unchanged, but for testing purposes
this produces more consistent results between WR and non-WR. It also seems
conceptually more correct so I think it's worth doing.

Depends on D74789

Last try push revealed that having two nsDisplayScrollInfo items for the same frame is bad news. I could hack around it by using the PerFrameKey mechanism, but it's better to just avoid creating the second one. With that fixed, I expect this try push to be green:

https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=224198a009aed8bf71ede8b33212bd7e55b80c79

Backed out 2 changesets (bug 1634763) for mochitest failures at gfx/layers/apz/test/mochitest/test_layerization.html

Backout: https://hg.mozilla.org/integration/autoland/rev/ab68fe76bdf468a3ee1b250e2d78e2a4d5422cbb

Failure push: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=f6a104ccde95542c420d016e34f88118cfc4c287&selectedTaskRun=Mz8KHbZARHSFpycV8Q54mQ-0

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=301880501&repo=autoland&lineNumber=14369

Thanks for the backout. I guess the way test_layerization.html is testing for layerization doesn't quite work with these changes.

I gave this a bit of thought but I'm having some doubts about what's supposed to happen conceptually. I think my patches might be creating too many scrollinfo layers - like, pretty much every inactive scrollframe might be getting one. I don't know if that's the only way to resolve the root problem here, or if I can reduce the set of inactive scrollframes that get scrollinfos.

Like, if every inactive scrollframe had this problem I would have expected to see this assertion fire a lot more. But not many people run debug builds and our test coverage misses surprisingly common things sometimes so I'm not sure.

Maybe putting the scrollid on the scrollbars should be conditional on mWillBuildScrollableLayer and not IsMaybeScrollingActive? So, still layerize scrollbars if IsMaybeScrollingActive but make the scrollid setting conditional on mWillBuildScrollableLayer?

Ah, that's a good idea, I hadn't considered that approach. I'll see if that works.

So that does fix the assertion and passes all the mochitests, but makes a few reftests fail. I haven't investigated yet. It could be that I didn't cancel out the effects of my previous patches well enough in the try push.

https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=161c2a095463ffade7cdf5fd9965c74a28d84f91

The severity field is not set for this bug.
:jbonisteel, could you have a look please?

For more information, please visit auto_nag documentation.

Delayed update: the approach in comment 22 didn't really pan out. I should have written down details when they were in mind but I think changes along those lines were significantly impacting layerization, because the slider frame uses the scroll id to determine layerization here and I didn't feel comfortable making changes to that code.

Also during APZ planning we discussed bug 1542057 which requires more inactive scrollframes getting scrollinfo layers, so I was thinking maybe it wouldn't be so bad if we just had all inactive scrollframes get them. It would certainly make fixing that bug much simpler.

There are some r+ patches which didn't land and no activity in this bug for 2 weeks.
:kats, could you have a look please?
For more information, please visit auto_nag documentation.

Yeah I'm gonna get back to this soonish.

Naively updated the patches her to trunk. They fail the added test.

I run into this a lot when using the JS debugger in my debug builds. Is there value in keeping the assert firing until we have a fix, or could we consider disabling?

(In reply to Bryce Seager van Dyk (:bryce) from comment #30)

I run into this a lot when using the JS debugger in my debug builds. Is there value in keeping the assert firing until we have a fix, or could we consider disabling?

Bryce, do you experience this when running with Fission enabled as well?

Based on reading through the comments in this bug, I would expect that work done in bug 1675547 should have resolved this problem in Fission mode.

Thanks for the heads up. I must admit I don't always remember to pass --enable-fission on my ./mach run invocations, so it's possible I'm only hitting this when I forget to do so. I'll be mindful to use the flag, and will follow up with another comment if I run into this with Fission on.

Sounds good! And if, for whatever reason, you need to run some things in non-Fission mode and the assertion is interfering, I'd definitely be supportive of downgrading it to NS_ASSERTION in non-Fission mode.

Can this be looked into? It's very frequency crashing debug builds and slows down development.

(In reply to Paul Adenot (:padenot) from comment #34)

Can this be looked into? It's very frequency crashing debug builds and slows down development.

Are you running in Fission mode or non-Fission mode when you run into this?

If you only see it in non-Fission mode, I think we can just downgrade it to NS_ASSERTION (in non-Fission mode only) as per comment 33.

(In reply to Kartikaya Gupta (email:kats@mozilla.staktrace.com) from comment #26)

Yeah I'm gonna get back to this soonish.

Well that was a lie. I'm going to unassign this bug since I doubt I'll get around to it.

(In reply to Botond Ballo [:botond] from comment #35)

Are you running in Fission mode or non-Fission mode when you run into this?

Non-fission, this was when running tests.

Thanks. Let's dowgrade the assertion in non-Fission mode then.

I believe the cause of the assertion is understood, and the proper fix is to take the Fission codepath which activates all scroll frames.

This assertion is known to affect some scenarios involving inactive
scroll frames (see comment 10-11 in this bug). In Fission mode, this
is resolved by making all scroll frames active. In non-Fission mode,
we don't plan to pursue a specific fix so downgrade the assertion
to avoid annoyance for users running debug builds.

https://hg.mozilla.org/mozilla-central/rev/2d8d58edaaa3