Found while fuzzing m-c 20230519-d97636946466 (--enable-debug --enable-fuzzing)

To reproduce via Grizzly Replay:

$ pip install fuzzfetch grizzly-framework
$ python -m fuzzfetch -d --fuzzing --cpu x86 -n firefox
$ python -m grizzly.replay ./firefox/firefox testcase.html

Assertion failure: false (MOZ_ASSERT_UNREACHABLE: unexpected to be called), at /builds/worker/checkouts/gecko/gfx/layers/RemoteTextureMap.cpp:667

6|0|xul.dll|mozilla::layers::RemoteTextureMap::GetRemoteTextureForDisplayList(mozilla::layers::RemoteTextureHostWrapper*, std::function<void (const mozilla::layers::RemoteTextureInfo &)>&&)|hg:hg.mozilla.org/mozilla-central:gfx/layers/RemoteTextureMap.cpp:a9e342522ebdb9602671208a06e988a285d3af18|667|0x711
6|1|xul.dll|mozilla::layers::WebRenderImageHost::UseRemoteTexture()|hg:hg.mozilla.org/mozilla-central:gfx/layers/wr/WebRenderImageHost.cpp:a9e342522ebdb9602671208a06e988a285d3af18|220|0x2bc
6|2|xul.dll|mozilla::layers::CompositableParentManager::ReceiveCompositableUpdate(mozilla::layers::CompositableOperationDetail const&, mozilla::NotNull<mozilla::layers::CompositableHost *>, mozilla::layers::CompositableHandle const&)|hg:hg.mozilla.org/mozilla-central:gfx/layers/ipc/CompositableTransactionParent.cpp:a9e342522ebdb9602671208a06e988a285d3af18|95|0x298
6|3|xul.dll|mozilla::layers::CompositableParentManager::ReceiveCompositableUpdate(mozilla::layers::CompositableOperation const&)|hg:hg.mozilla.org/mozilla-central:gfx/layers/ipc/CompositableTransactionParent.cpp:a9e342522ebdb9602671208a06e988a285d3af18|35|0x82
6|4|xul.dll|mozilla::layers::WebRenderBridgeParent::ProcessWebRenderParentCommands(nsTArray<mozilla::layers::WebRenderParentCommand> const&, mozilla::wr::TransactionBuilder&)|hg:hg.mozilla.org/mozilla-central:gfx/layers/wr/WebRenderBridgeParent.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1517|0x21b
6|5|xul.dll|mozilla::layers::WebRenderBridgeParent::ProcessDisplayListData(mozilla::layers::DisplayListData&, mozilla::wr::Epoch, mozilla::TimeStamp const&, bool, bool)|hg:hg.mozilla.org/mozilla-central:gfx/layers/wr/WebRenderBridgeParent.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1169|0xd9
6|6|xul.dll|mozilla::layers::WebRenderBridgeParent::RecvSetDisplayList(mozilla::layers::DisplayListData&&, nsTArray<mozilla::layers::OpDestroy>&&, unsigned long long const&, mozilla::layers::BaseTransactionId<mozilla::layers::TransactionIdType> const&, bool const&, mozilla::layers::BaseTransactionId<mozilla::VsyncIdType> const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, nsTSubstring<char> const&, mozilla::TimeStamp const&, nsTArray<mozilla::layers::CompositionPayload>&&)|hg:hg.mozilla.org/mozilla-central:gfx/layers/wr/WebRenderBridgeParent.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1233|0x23a
6|7|xul.dll|mozilla::layers::PWebRenderBridgeParent::OnMessageReceived(IPC::Message const&)|s3:gecko-generated-sources:777c843b67635508ddfb6f2f0da1fcd0804deebbce07f33d176dfc439d033df5d067223ae92833d61dbe5a69d74b75e53f8a48e300fe4cf7f528fed615b288a2/ipc/ipdl/PWebRenderBridgeParent.cpp:|489|0x1995
6|8|xul.dll|mozilla::layers::PCompositorManagerParent::OnMessageReceived(IPC::Message const&)|s3:gecko-generated-sources:8a5c9cc165d7d668ca5410663e4f25e5dc6ad7d155ae9883734d6883bbd6597f509d7c66e9fdd7365d896d91f348d7e3de10400d7c699126ef48a3442327480e/ipc/ipdl/PCompositorManagerParent.cpp:|194|0x28c
6|9|xul.dll|mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1811|0x128
6|10|xul.dll|mozilla::ipc::MessageChannel::DispatchMessage(mozilla::ipc::ActorLifecycleProxy*, mozilla::UniquePtr<IPC::Message,mozilla::DefaultDelete<IPC::Message> >)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1736|0x1f3
6|11|xul.dll|mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::ActorLifecycleProxy*, mozilla::ipc::MessageChannel::MessageTask&)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1536|0x155
6|12|xul.dll|mozilla::ipc::MessageChannel::MessageTask::Run()|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessageChannel.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1634|0xba
6|13|xul.dll|nsThread::ProcessNextEvent(bool, bool*)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThread.cpp:a9e342522ebdb9602671208a06e988a285d3af18|1193|0x9ae
6|14|xul.dll|NS_ProcessNextEvent(nsIThread*, bool)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThreadUtils.cpp:a9e342522ebdb9602671208a06e988a285d3af18|479|0x41
6|15|xul.dll|mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*)|hg:hg.mozilla.org/mozilla-central:ipc/glue/MessagePump.cpp:a9e342522ebdb9602671208a06e988a285d3af18|300|0xe0
6|16|xul.dll|MessageLoop::RunInternal()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:a9e342522ebdb9602671208a06e988a285d3af18|370|0x82
6|17|xul.dll|MessageLoop::RunHandler()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:a9e342522ebdb9602671208a06e988a285d3af18|363|0x72
6|18|xul.dll|MessageLoop::Run()|hg:hg.mozilla.org/mozilla-central:ipc/chromium/src/base/message_loop.cc:a9e342522ebdb9602671208a06e988a285d3af18|345|0x55
6|19|xul.dll|nsThread::ThreadFunc(void*)|hg:hg.mozilla.org/mozilla-central:xpcom/threads/nsThread.cpp:a9e342522ebdb9602671208a06e988a285d3af18|391|0x18e
6|20|nss3.dll|_PR_NativeRunThread(void*)|hg:hg.mozilla.org/mozilla-central:nsprpub/pr/src/threads/combined/pruthr.c:a9e342522ebdb9602671208a06e988a285d3af18|399|0xe3
6|21|nss3.dll|pr_root(void*)|hg:hg.mozilla.org/mozilla-central:nsprpub/pr/src/md/windows/w95thred.c:a9e342522ebdb9602671208a06e988a285d3af18|139|0x15
6|22|ucrtbase.dll||||
6|23|kernel32.dll||||
6|24|mozglue.dll|patched_BaseThreadInitThunk(int, void*, void*)|hg:hg.mozilla.org/mozilla-central:toolkit/xre/dllservices/mozglue/WindowsDllBlocklist.cpp:a9e342522ebdb9602671208a06e988a285d3af18|617|0x72
6|25|ntdll.dll||||
6|26|ntdll.dll||||

Sotaro, is this a part of the code that you would know about?

Bugmon is currently only supported on Linux.

:tsmith, does the problem still happen with latest nightly?

I do not have an environment of "--cpu x86", but tried with "python -m fuzzfetch -d --fuzzing -n firefox" on Linux, the problem did not happen. And testcase.html did not hit assert failure with local build.

(In reply to Sotaro Ikeda [:sotaro] from comment #3)

:tsmith, does the problem still happen with latest nightly?

Yes this is reproducible with m-c 20230628-3b7b3970a884.

I do not have an environment of "--cpu x86"

A 32-bit x86 build just works on Windows x64_86. So all you need is a machine running Windows . I am not able to reproduce this on Linux.

I looked into source code, ClientWebGLContext, WebGLParent and WebGLContext handle the default frame buffer size as expected. Then it seemed that an unexpected error caused the problem.

• when size of WebGL canvas is resized, ClientWebGLContext::SetDimensions() is called, it set ClientWebGLContext::mResetLayer to true. And WebGLParent is requested to resize by sync for async way.
• When ClientWebGLContext::UpdateWebRenderCanvasData() is called, mResetLayer = true requests to call ClientWebGLContext::DrawingBufferSize(). child->SendDrawingBufferSize() is sync IPC. It end up to call WebGLContext::DrawingBufferSize()
• WebGLContext::DrawingBufferSize() calls EnsureDefaultFB(). WebGLContext::EnsureDefaultFB() allocates default frame buffer when it is not allocated yet.

Therefore ClientWebGLContext::UpdateWebRenderCanvasData() get actual valid default frame buffer size from WebGLContext.

I succeeded to reproduce the assertion failure with Attachment 9339917 [details] and with local build(64bit build) only once. When it happened, Pbuffer surface allocation was failed at CreatePBufferSurface().

Actually Comment 6 is a problem of ANGLE.

:tsmith, can you get logout to console with With pref security.sandbox.gpu.level = 0? It seemed that several error might happen before the assertion failure.

Thank you for the log! From the following, Texture2D allocation was failed in ANGLE and device reset happened. Then WebGL context was lost.

allocate:404. Internal D3D11 error: HRESULT: 0x887A0005 (removal reason: HRESULT: 0x887A0020): Error allocating Texture2D
WebGL(07388900)::LoseContext(0)

bug 1841214 seems to address the problem.

:tsmith, can you check if the problem is addressed?

(In reply to Sotaro Ikeda [:sotaro] from comment #11)

:tsmith, can you check if the problem is addressed?

The attached test case now triggers bug 1845554.

With Bug 1841214 fix, a problem that is related to RemoteTexture is addressed.
Since bug 1845554 exists, this bug could be closed.