Filed by: smolnar [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=409841872&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/fufuCIPlTh-eaVBITinW9A/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/fufuCIPlTh-eaVBITinW9A/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

INFO - TEST-START | /_mozilla/webgpu/webgpu/web_platform/reftests/canvas_colorspace_bgra8unorm.https.html
[task 2023-03-22T22:16:57.954Z] 22:16:57     INFO - PID 4120 | [Parent 9772, Main Thread] WARNING: NS_ENSURE_TRUE(uri) failed: file /builds/worker/checkouts/gecko/caps/BasePrincipal.cpp:1495
[task 2023-03-22T22:16:58.010Z] 22:16:58     INFO - PID 4120 | [WARN  webrender::device::gl] Missing optimized shader source for gpu_cache_update
[task 2023-03-22T22:16:58.035Z] 22:16:58     INFO - PID 4120 | [WARN  webrender::device::gl] Cropping texture upload Box2D((0, 0), (0, 1)) to None
[task 2023-03-22T22:16:58.036Z] 22:16:58     INFO - PID 4120 | [WARN  webrender::device::gl] Cropping texture upload Box2D((0, 0), (0, 1)) to None
[task 2023-03-22T22:16:58.037Z] 22:16:58     INFO - PID 4120 | [WARN  webrender::device::gl] Cropping texture upload Box2D((0, 0), (0, 1)) to None
[task 2023-03-22T22:16:58.037Z] 22:16:58     INFO - PID 4120 | [WARN  webrender::device::gl] Cropping texture upload Box2D((0, 0), (0, 1)) to None
[task 2023-03-22T22:16:58.038Z] 22:16:58     INFO - PID 4120 | [WARN  webrender::device::gl] Cropping texture upload Box2D((0, 0), (0, 1)) to None
[task 2023-03-22T22:16:58.038Z] 22:16:58     INFO - PID 4120 | [WARN  webrender::device::gl] Cropping texture upload Box2D((0, 0), (0, 1)) to None
[task 2023-03-22T22:16:58.039Z] 22:16:58     INFO - PID 4120 | [WARN  webrender::device::gl] Cropping texture upload Box2D((0, 0), (0, 1)) to None
[task 2023-03-22T22:16:58.040Z] 22:16:58     INFO - PID 4120 | [WARN  webrender::device::gl] Cropping texture upload Box2D((0, 0), (0, 1)) to None
[task 2023-03-22T22:16:58.041Z] 22:16:58     INFO - PID 4120 | [WARN  webrender::device::gl] Cropping texture upload Box2D((0, 0), (0, 1)) to None
[task 2023-03-22T22:16:58.062Z] 22:16:58     INFO - PID 4120 | 1679523418061	Marionette	INFO	Testing https://web-platform.test:8443/_mozilla/webgpu/webgpu/web_platform/reftests/canvas_colorspace_bgra8unorm.https.html == http://web-platform.test:8000/_mozilla/webgpu/webgpu/web_platform/reftests/ref/canvas_colorspace-ref.html
[task 2023-03-22T22:16:58.109Z] 22:16:58     INFO - PID 4120 | [Child 2812, Main Thread] WARNING: Extra shutdown CC: 'i < NORMAL_SHUTDOWN_COLLECTIONS', file /builds/worker/checkouts/gecko/xpcom/base/nsCycleCollector.cpp:3396
[task 2023-03-22T22:16:58.111Z] 22:16:58     INFO - PID 4120 | [Child 2812, Main Thread] WARNING: NS_ENSURE_TRUE(InitStaticMembers()) failed: file /builds/worker/workspace/obj-build/dist/include/mozilla/Preferences.h:129
[task 2023-03-22T22:16:58.119Z] 22:16:58     INFO - PID 4120 | [Child 2812, Main Thread] WARNING: NS_ENSURE_TRUE(Preferences::InitStaticMembers()) failed: file /builds/worker/checkouts/gecko/modules/libpref/Preferences.cpp:4689
[task 2023-03-22T22:16:58.342Z] 22:16:58     INFO - PID 4120 | [GPU 3316, BackgroundThreadPool #1] WARNING: Failed to lock the texture: file /builds/worker/checkouts/gecko/gfx/layers/d3d11/TextureD3D11.cpp:251
[task 2023-03-22T22:16:58.383Z] 22:16:58     INFO - PID 4120 | [Child 9324, Main Thread] WARNING: Extra shutdown CC: 'i < NORMAL_SHUTDOWN_COLLECTIONS', file /builds/worker/checkouts/gecko/xpcom/base/nsCycleCollector.cpp:3396
[task 2023-03-22T22:16:58.384Z] 22:16:58     INFO - PID 4120 | [Child 9324, Main Thread] WARNING: NS_ENSURE_TRUE(InitStaticMembers()) failed: file /builds/worker/workspace/obj-build/dist/include/mozilla/Preferences.h:129
[task 2023-03-22T22:16:58.393Z] 22:16:58     INFO - PID 4120 | [Child 9324, Main Thread] WARNING: NS_ENSURE_TRUE(Preferences::InitStaticMembers()) failed: file /builds/worker/checkouts/gecko/modules/libpref/Preferences.cpp:4689
[task 2023-03-22T22:16:58.685Z] 22:16:58     INFO - PID 4120 | 1679523418683	Marionette	INFO	Emitted TestRendered event
[task 2023-03-22T22:16:58.689Z] 22:16:58     INFO - PID 4120 | [WARN  wgpu_hal::vulkan::instance] Unable to find layer: VK_LAYER_KHRONOS_validation
[task 2023-03-22T22:16:58.708Z] 22:16:58     INFO - PID 4120 | [WARN  wgpu_hal::dx12::instance] Unable to enable D3D12 debug interface: 0x887A002D
[task 2023-03-22T22:16:58.709Z] 22:16:58     INFO - PID 4120 | [WARN  wgpu_hal::auxil::dxgi::factory] Unable to enable DXGI debug interface: 0x887A002D
[task 2023-03-22T22:16:58.846Z] 22:16:58     INFO - PID 4120 | 1679523418845	Marionette	INFO	No differences allowed
[task 2023-03-22T22:16:58.934Z] 22:16:58     INFO - TEST-UNEXPECTED-FAIL | /_mozilla/webgpu/webgpu/web_platform/reftests/canvas_colorspace_bgra8unorm.https.html | Testing https://web-platform.test:8443/_mozilla/webgpu/webgpu/web_platform/reftests/canvas_colorspace_bgra8unorm.https.html == http://web-platform.test:8000/_mozilla/webgpu/webgpu/web_platform/reftests/ref/canvas_colorspace-ref.html
[task 2023-03-22T22:16:58.934Z] 22:16:58     INFO - Found 65536 pixels different, maximum difference per channel 255
<...>
INFO - TEST-INFO expected TIMEOUT | took 961ms
[task 2023-03-22T22:16:58.943Z] 22:16:58     INFO - PID 4120 | [GPU 3316, Compositor] WARNING: Possibly dropping task posted to updater thread: file /builds/worker/checkouts/gecko/gfx/layers/apz/src/APZUpdater.cpp:371
[task 2023-03-22T22:16:58.986Z] 22:16:58     INFO - PID 4120 | 1679523418984	Marionette	INFO	Stopped listening on port 58812
[task 2023-03-22T22:16:59.018Z] 22:16:59     INFO - PID 4120 | [Child 2100, Main Thread] WARNING: Extra shutdown CC: 'i < NORMAL_SHUTDOWN_COLLECTIONS', file /builds/worker/checkouts/gecko/xpcom/base/nsCycleCollector.cpp:3396
[task 2023-03-22T22:16:59.021Z] 22:16:59     INFO - PID 4120 | [Child 2100, Main Thread] WARNING: NS_ENSURE_TRUE(InitStaticMembers()) failed: file /builds/worker/workspace/obj-build/dist/include/mozilla/Preferences.h:129
[task 2023-03-22T22:16:59.029Z] 22:16:59     INFO - PID 4120 | [Child 2100, Main Thread] WARNING: NS_ENSURE_TRUE(Preferences::InitStaticMembers()) failed: file /builds/worker/checkouts/gecko/modules/libpref/Preferences.cpp:4689
[task 2023-03-22T22:16:59.035Z] 22:16:59     INFO - PID 4120 | [Child 9368, Main Thread] WARNING: '!CanSend() || !mManager || !mManager->CanSend()', file /builds/worker/checkouts/gecko/dom/ipc/jsactor/JSWindowActorChild.cpp:44
[task 2023-03-22T22:16:59.036Z] 22:16:59     INFO - PID 4120 | [Child 9368, Main Thread] WARNING: '!CanSend() || !mManager || !mManager->CanSend()', file /builds/worker/checkouts/gecko/dom/ipc/jsactor/JSWindowActorChild.cpp:44
[task 2023-03-22T22:16:59.038Z] 22:16:59     INFO - PID 4120 | [Child 9368, Main Thread] WARNING: '!CanSend() || !mManager || !mManager->CanSend()', file /builds/worker/checkouts/gecko/dom/ipc/jsactor/JSWindowActorChild.cpp:44
[task 2023-03-22T22:16:59.044Z] 22:16:59     INFO - PID 4120 | [Child 9368, Main Thread] WARNING: '!CanSend() || !mManager || !mManager->CanSend()', file /builds/worker/checkouts/gecko/dom/ipc/jsactor/JSWindowActorChild.cpp:44
[task 2023-03-22T22:16:59.046Z] 22:16:59     INFO - PID 4120 | [Child 9368, Main Thread] WARNING: '!CanSend() || !mManager || !mManager->CanSend()', file /builds/worker/checkouts/gecko/dom/ipc/jsactor/JSWindowActorChild.cpp:44
[task 2023-03-22T22:16:59.047Z] 22:16:59     INFO - PID 4120 | [Child 9368, Main Thread] WARNING: '!CanSend() || !mManager || !mManager->CanSend()', file /builds/worker/checkouts/gecko/dom/ipc/jsactor/JSWindowActorChild.cpp:44
[task 2023-03-22T22:16:59.056Z] 22:16:59     INFO - PID 4120 | [Child 9368, Main Thread] WARNING: '!CanSend() || !mManager || !mManager->CanSend()', file /builds/worker/checkouts/gecko/dom/ipc/jsactor/JSWindowActorChild.cpp:44
[task 2023-03-22T22:16:59.138Z] 22:16:59     INFO - PID 4120 | [GPU 3316, Compositor] WARNING: Possibly dropping task posted to updater thread: file /builds/worker/checkouts/gecko/gfx/layers/apz/src/APZUpdater.cpp:371
[task 2023-03-22T22:16:59.185Z] 22:16:59     INFO - PID 4120 | [Parent 9772, Socket Thread] WARNING: cannot post event if not initialized: file /builds/worker/checkouts/gecko/netwerk/protocol/http/nsHttpConnectionMgr.cpp:232
[task 2023-03-22T22:16:59.186Z] 22:16:59     INFO - PID 4120 | [Parent 9772, Socket Thread] WARNING: cannot post event if not initialized: file /builds/worker/checkouts/gecko/netwerk/protocol/http/nsHttpConnectionMgr.cpp:232
[task 2023-03-22T22:16:59.207Z] 22:16:59     INFO - PID 4120 | [Child 5964, Main Thread] WARNING: Extra shutdown CC: 'i < NORMAL_SHUTDOWN_COLLECTIONS', file /builds/worker/checkouts/gecko/xpcom/base/nsCycleCollector.cpp:3396
[task 2023-03-22T22:16:59.210Z] 22:16:59     INFO - PID 4120 | [Child 5964, Main Thread] WARNING: NS_ENSURE_TRUE(InitStaticMembers()) failed: file /builds/worker/workspace/obj-build/dist/include/mozilla/Preferences.h:129
[task 2023-03-22T22:16:59.219Z] 22:16:59     INFO - PID 4120 | [Child 5964, Main Thread] WARNING: NS_ENSURE_TRUE(Preferences::InitStaticMembers()) failed: file /builds/worker/checkouts/gecko/modules/libpref/Preferences.cpp:4689
[task 2023-03-22T22:16:59.264Z] 22:16:59     INFO - PID 4120 | [Child 9368, Main Thread] WARNING: Extra shutdown CC: 'i < NORMAL_SHUTDOWN_COLLECTIONS', file /builds/worker/checkouts/gecko/xpcom/base/nsCycleCollector.cpp:3396
[task 2023-03-22T22:16:59.266Z] 22:16:59     INFO - PID 4120 | [Child 9368, Main Thread] WARNING: NS_ENSURE_TRUE(InitStaticMembers()) failed: file /builds/worker/workspace/obj-build/dist/include/mozilla/Preferences.h:129
[task 2023-03-22T22:16:59.275Z] 22:16:59     INFO - PID 4120 | [Child 9368, Main Thread] WARNING: NS_ENSURE_TRUE(Preferences::InitStaticMembers()) failed: file /builds/worker/checkouts/gecko/modules/libpref/Preferences.cpp:4689
[task 2023-03-22T22:16:59.501Z] 22:16:59     INFO - PID 4120 | [Parent 9772, IPDL Background] WARNING: quota manager shutdown step: '0.000000s: startCrashBrowserTimer', file /builds/worker/checkouts/gecko/dom/quota/ActorsParent.cpp:3330
[task 2023-03-22T22:16:59.502Z] 22:16:59     INFO - PID 4120 | [Parent 9772, IPDL Background] WARNING: quota manager shutdown step: '0.000000s: initiateShutdownWorkThreads', file /builds/worker/checkouts/gecko/dom/quota/ActorsParent.cpp:3330
[task 2023-03-22T22:16:59.503Z] 22:16:59     INFO - PID 4120 | [Parent 9772, IPDL Background] WARNING: idb: '0.000000s: starting', file /builds/worker/checkouts/gecko/dom/quota/ActorsParent.cpp:3330
[task 2023-03-22T22:16:59.504Z] 22:16:59     INFO - PID 4120 | [Parent 9772, IPDL Background] WARNING: cache: '0.000000s: starting', file /builds/worker/checkouts/gecko/dom/quota/ActorsParent.cpp:3330
[task 2023-03-22T22:16:59.504Z] 22:16:59     INFO - PID 4120 | [Parent 9772, IPDL Background] WARNING: sdb: '0.000000s: starting', file /builds/worker/checkouts/gecko/dom/quota/ActorsParent.cpp:3330
[task 2023-03-22T22:16:59.505Z] 22:16:59     INFO - PID 4120 | [Parent 9772, IPDL Background] WARNING: fs: '0.000000s: starting', file /builds/worker/checkouts/gecko/dom/quota/ActorsParent.cpp:3330
[task 2023-03-22T22:16:59.505Z] 22:16:59     INFO - PID 4120 | [Parent 9772, IPDL Background] WARNING: ls: '0.000000s: starting', file /builds/worker/checkouts/gecko/dom/quota/ActorsParent.cpp:3330

Set release status flags based on info from the regressing bug 1720941

Will examine this issue in my day tomorrow.

Silencing the failure, but leaving the bug open to track the work, since we do want to re-enable this test eventually.

https://hg.mozilla.org/mozilla-central/rev/72fc4b6a6b55

As evidenced by a recent Try push, my workaround solution for bug 1824042 was wrong for a couple of reasons:

1. os == "windows" aggravatingly works for me locally, but not in CI.
2. Squashing the crashes in the offending tests noted in the original workaround solution merely moved the crashing behavior's effects to other tests.

I think it's safe to say that we have a more complicated issue than a single reftest here. So...let's just disable these reftests on windows win for now.

https://hg.mozilla.org/mozilla-central/rev/a547406d4b1f

Info dump from my timeboxed attempts to diagnose this:

• Erich's current hypothesis (chat link): This might be wgpu#3485.

• Maybe this line about texture locking is relevant?

  WARNING: Failed to lock the texture: file /builds/worker/checkouts/gecko/gfx/layers/d3d11/TextureD3D11.cpp:251
  

• This seems to happen exclusively on 64-bit builds. 32-bit builds do not reproduce this issue. 👀

• Optimized builds are not crashing. 👀

• 64-bit Windows 11 runner info from wgpu:

  [INFO  wgpu_core::instance] Adapter Dx12 AdapterInfo { name: "Microsoft Basic Render Driver", vendor: 5140, device: 140, device_type: IntegratedGpu, driver: "", driver_info: "", backend: Dx12 }
  

  From WebGL:

  GL_RENDERER: ANGLE (NVIDIA, NVIDIA Tesla M60 Direct3D11 vs_5_0 ps_5_0, D3D11-30.0.14.7239)
  mRenderer: Unknown
  

• :jrmuizel has suggested that disabling attempts to use the D3D debug interface might be helpful.

• Still need to make bug repro. a short feedback loop.

  • Maybe one could locally edit to only pick a "bad" adapter in wgpu, rather than the known working discrete GPU? That might not work, though, because the offending failures look like they use an integrated GPU.
  • Get access to a machine that reproduces this problem to shorten the CI feedback loop.
    • Use an interactive task in CI?
    • Asked Joel Maher about getting access to one of the machines repro'ing failures, got forwarded to Jonathan Moss (who hasn't responded yet).
    • See who else has Maxwell-architecture machines
      • :jgilbert appears to have a few Maxwell-architecture machines, which :jrmuizel states are probably similar to the 64-bit Windows 11 runners. Asked, awaiting response.

ErichDonGubler: I will get a loaner setup early next week. Who will need access to it? Just to verified the type needed is gecko-t/win11-64-2009-gpu?

For future reference, the best avenue for requests for ReleaseSRE would be the Slack Relops channel, a Relops Jira ticket or Bugzilla bug.

:markco: Getting :jimb and myself access to it should be sufficient for now. Thank you!

Set release status flags based on info from the regressing bug 1720941

I have a loaner setup. How would you like to receive the username/password?

:markco: Receiving via your choice of work email, Slack, or Matrix would be great. Thank you!

The info has been sent.

Received, thanks, :markco! I'm going to check that I can actually access the machine shortly, but I may not return to this task for a week or so. I understand that there is a window of a few weeks (plus the option for extending), so I'm not overly concerned.

I've validated that I have access to a machine with the expected Maxwell generation of Nvidia hardware (Tesla M60). Thanks again, :markco!

Took some time to investigate this today on the machine provided by :markco. My findings are basically that bgra8unorm and rgba16float texture format support should remove blockers in each of the failing test cases right now. This particular test case is failing because our WebGPU implementation currently lacks of bgra8unorm support. If we were to support bgra8unorm and rgba16float for currently failing WebGPU reftests, we would remove those specific blockers. There may be more beyond the blockers we're seeing, but we'll need to handle those with separate bugs when we find them.

I've filed a ticket for rgba16float support specifically (bug 1834395). Duping this one into bug 1829895.