08:04:36 WARNING - PROCESS-CRASH | dom/media/tests/mochitest/test_peerConnection_captureStream_canvas_2d.html | application crashed [@ nvwgf2umx.dll + 0xef889] 08:04:36 INFO - Crash dump filename: c:\users\cltbld~1.t-w\appdata\local\temp\tmpexfpwt.mozrunner\minidumps\d216daf1-61f0-4c8d-9219-7b980c44dc14.dmp 08:04:36 INFO - Operating system: Windows NT 08:04:36 INFO - 6.2.9200 08:04:36 INFO - CPU: amd64 08:04:36 INFO - family 6 model 30 stepping 5 08:04:36 INFO - 8 CPUs 08:04:36 INFO - Crash reason: EXCEPTION_ACCESS_VIOLATION_READ 08:04:36 INFO - Crash address: 0x4 08:04:36 INFO - Thread 28 (crashed) 08:04:36 INFO - 0 nvwgf2umx.dll + 0xef889 08:04:36 INFO - rbx = 0x0000000000000000 r12 = 0x0000000000000000 08:04:36 INFO - r13 = 0x0000000000000000 r14 = 0x0000000000000000 08:04:36 INFO - r15 = 0x000000845c5a7a58 rip = 0x000007ff1456f889 08:04:36 INFO - rsp = 0x000000845c14f9b0 rbp = 0x0000000000000000 08:04:36 INFO - Found by: given as instruction pointer in context 08:04:36 INFO - 1 nvwgf2umx.dll + 0x2d0736 08:04:36 INFO - rip = 0x000007ff14750737 rsp = 0x000000845c14f9c0 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 2 nvwgf2umx.dll + 0x2c07af 08:04:36 INFO - rip = 0x000007ff147407b0 rsp = 0x000000845c14f9f0 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 3 nvwgf2umx.dll + 0xaeafaf 08:04:36 INFO - rip = 0x000007ff14f6afb0 rsp = 0x000000845c14fa60 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 4 nvwgf2umx.dll + 0x33f4de 08:04:36 INFO - rip = 0x000007ff147bf4df rsp = 0x000000845c14faf0 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 5 nvwgf2umx.dll + 0x193ff 08:04:36 INFO - rip = 0x000007ff14499400 rsp = 0x000000845c14fb50 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 6 nvwgf2umx.dll + 0xaeafaf 08:04:36 INFO - rip = 0x000007ff14f6afb0 rsp = 0x000000845c14fb68 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 7 nvwgf2umx.dll + 0x3fc8df 08:04:36 INFO - rip = 0x000007ff1487c8e0 rsp = 0x000000845c14fb90 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 8 nvwgf2umx.dll + 0xaeafaf 08:04:36 INFO - rip = 0x000007ff14f6afb0 rsp = 0x000000845c14fbc8 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 9 nvwgf2umx.dll + 0xb023f 08:04:36 INFO - rip = 0x000007ff14530240 rsp = 0x000000845c14fc00 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 10 nvwgf2umx.dll + 0x2cf1ae 08:04:36 INFO - rip = 0x000007ff1474f1af rsp = 0x000000845c14fc10 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 11 nvwgf2umx.dll + 0xadbe3 08:04:36 INFO - rip = 0x000007ff1452dbe4 rsp = 0x000000845c14fc70 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 12 nvwgf2umx.dll + 0xa9c4c 08:04:36 INFO - rip = 0x000007ff14529c4d rsp = 0x000000845c14fca0 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 13 nvwgf2umx.dll + 0x1d5dc9 08:04:36 INFO - rip = 0x000007ff14655dca rsp = 0x000000845c14fcd0 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 14 nvwgf2umx.dll + 0x9e02c6 08:04:36 INFO - rip = 0x000007ff14e602c7 rsp = 0x000000845c14fd00 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 15 nvwgf2umx.dll + 0x9e046d 08:04:36 INFO - rip = 0x000007ff14e6046e rsp = 0x000000845c14fd30 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 16 kernel32.dll + 0x167d 08:04:36 INFO - rip = 0x000007ff1b8d167e rsp = 0x000000845c14fd60 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 17 ntdll.dll + 0x1c3f0 08:04:36 INFO - rip = 0x000007ff1bf3c3f1 rsp = 0x000000845c14fd90 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 18 KERNELBASE.dll + 0x809cf 08:04:36 INFO - rip = 0x000007ff192b09d0 rsp = 0x000000845c14fdc0 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - Thread 0 08:04:36 INFO - 0 ntdll.dll + 0x2bea 08:04:36 INFO - rbx = 0x000000845c5944d0 r12 = 0x0000000000000000 08:04:36 INFO - r13 = 0x0000000000000000 r14 = 0x000007f72ad7e000 08:04:36 INFO - r15 = 0x000000007ffe0382 rip = 0x000007ff1bf22bea 08:04:36 INFO - rsp = 0x000000844ebfdf28 rbp = 0x0000000000000000 08:04:36 INFO - Found by: given as instruction pointer in context 08:04:36 INFO - 1 ntdll.dll + 0x19d29 08:04:36 INFO - rip = 0x000007ff1bf39d2a rsp = 0x000000844ebfdf30 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 2 KERNELBASE.dll + 0x1a21 08:04:36 INFO - rip = 0x000007ff19231a22 rsp = 0x000000844ebfdf60 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 3 nvwgf2umx.dll + 0xaef84 08:04:36 INFO - rip = 0x000007ff1452ef85 rsp = 0x000000844ebfdf90 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 4 nvwgf2umx.dll + 0xb7c54 08:04:36 INFO - rip = 0x000007ff14537c55 rsp = 0x000000844ebfdfc0 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 5 ntdll.dll + 0x103b3 08:04:36 INFO - rip = 0x000007ff1bf303b4 rsp = 0x000000844ebfe000 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 6 d3d11.dll + 0x4884c 08:04:36 INFO - rip = 0x000007ff15a0884d rsp = 0x000000844ebfe040 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 7 d3d11.dll + 0x5bb5b 08:04:36 INFO - rip = 0x000007ff15a1bb5c rsp = 0x000000844ebfe070 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 8 d3d11.dll + 0x3dc6c 08:04:36 INFO - rip = 0x000007ff159fdc6d rsp = 0x000000844ebfe090 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 9 d3d11.dll + 0x16219f 08:04:36 INFO - rip = 0x000007ff15b221a0 rsp = 0x000000844ebfe0d0 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 10 d3d11.dll + 0x5ba90 08:04:36 INFO - rip = 0x000007ff15a1ba91 rsp = 0x000000844ebfe120 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 11 xul.dll!mozilla::layers::SyncObjectD3D11::FinalizeFrame() [TextureD3D11.cpp:7d119f9b1e10 : 1116 + 0xc] 08:04:36 INFO - rip = 0x000007fef9ea6024 rsp = 0x000000844ebfe150 08:04:36 INFO - Found by: stack scanning 08:04:36 INFO - 12 xul.dll!mozilla::layers::ClientLayerManager::ForwardTransaction(bool) [ClientLayerManager.cpp:7d119f9b1e10 : 550 + 0x5] 08:04:36 INFO - rip = 0x000007fef9e777a1 rsp = 0x000000844ebfe6a0 08:04:36 INFO - Found by: call frame info 08:04:36 INFO - 13 xul.dll!mozilla::layers::ClientLayerManager::EndEmptyTransaction(mozilla::layers::LayerManager::EndTransactionFlags) [ClientLayerManager.cpp:7d119f9b1e10 : 353 + 0x14] 08:04:36 INFO - rip = 0x000007fef9e763d9 rsp = 0x000000844ebfe960 08:04:36 INFO - Found by: call frame info 08:04:36 INFO - 14 xul.dll!PresShell::Paint(nsView *,nsRegion const &,unsigned int) [nsPresShell.cpp:7d119f9b1e10 : 6329 + 0x10] 08:04:36 INFO - rip = 0x000007fefba56f5e rsp = 0x000000844ebfe990 08:04:36 INFO - Found by: call frame info 08:04:36 INFO - 15 xul.dll!nsViewManager::ProcessPendingUpdatesPaint(nsIWidget *) [nsViewManager.cpp:7d119f9b1e10 : 456 + 0x24] 08:04:36 INFO - rip = 0x000007fefb6b0260 rsp = 0x000000844ebfeb10 08:04:36 INFO - Found by: call frame info 08:04:36 INFO - 16 xul.dll!nsViewManager::ProcessPendingUpdatesForView(nsView *,bool) [nsViewManager.cpp:7d119f9b1e10 : 396 + 0xb] 08:04:36 INFO - rip = 0x000007fefb6b009b rsp = 0x000000844ebfeb70 08:04:36 INFO - Found by: call frame info 08:04:36 INFO - 17 xul.dll!nsViewManager::ProcessPendingUpdates() [nsViewManager.cpp:7d119f9b1e10 : 1086 + 0xe] 08:04:36 INFO - rip = 0x000007fefb6afef2 rsp = 0x000000844ebfebe0 08:04:36 INFO - Found by: call frame info 08:04:36 INFO - 18 xul.dll!nsRefreshDriver::Tick(__int64,mozilla::TimeStamp) [nsRefreshDriver.cpp:7d119f9b1e10 : 1776 + 0x2e] 08:04:36 INFO - rip = 0x000007fefb97656e rsp = 0x000000844ebfec10 08:04:36 INFO - Found by: call frame info 08:04:36 INFO - 19 xul.dll!mozilla::RefreshDriverTimer::TickDriver(nsRefreshDriver *,__int64,mozilla::TimeStamp) [nsRefreshDriver.cpp:7d119f9b1e10 : 194 + 0x22] 08:04:36 INFO - rip = 0x000007fefb9767d3 rsp = 0x000000844ebfef80 08:04:36 INFO - Found by: call frame info 08:04:36 INFO - 20 xul.dll!mozilla::RefreshDriverTimer::Tick(__int64,mozilla::TimeStamp) [nsRefreshDriver.cpp:7d119f9b1e10 : 185 + 0x2d] 08:04:36 INFO - rip = 0x000007fefb975a02 rsp = 0x000000844ebfefd0 08:04:36 INFO - Found by: call frame info

This and bug 1165928 seem very-likely related, and both are spiking at the moment. Can we get a Windows gfx person to look into this please?

Andrew, Kyle, do you think this could be from your recent changes?

(In reply to Milan Sreckovic [:milan] from comment #5) > Andrew, Kyle, do you think this could be from your recent changes? Probably not. The changes we made are only applicable to machines running Windows 8.1, which I don't believe any of the try servers use.

I thought D2D1.1 is available on Windows 7 machines with an update? Can you take a quick look anyway? Maybe send to a try with a backed out patch for bug 1161642 and see if it makes a difference? It did start on the same day as that bug landed, which may very well be a coincidence, but it's worth checking.

The primitive blends feature is still only available on Windows 8.1. We do a version check here: https://dxr.mozilla.org/mozilla-central/source/gfx/2d/HelpersD2D.h#284 Here's a try push with the change reverted: https://treeherder.mozilla.org/#/jobs?repo=try&revision=20552b9708e9

Looking closer, Jeff, is this in the window manager? Could it be related to the whale work?

And adding Bas as well, as this is in FinalizeFrame, and maybe can help with bug 1160157?

(In reply to Milan Sreckovic [:milan] from comment #9) > Looking closer, Jeff, is this in the window manager? Could it be related to > the whale work? What did you see that prompted this question?

(In reply to Jeff Muizelaar [:jrmuizel] from comment #11) > (In reply to Milan Sreckovic [:milan] from comment #9) > > Looking closer, Jeff, is this in the window manager? Could it be related to > > the whale work? > > What did you see that prompted this question? Random thoughts and shots in the dark and reports somebody had dwm crashing in this driver after a windows update. So, nothing solid.

This was a test added by pehrsons (taking canvas data and pushing to a MediaStream). Though the crash certainly seems to be a GFX crash... Any traction here? Still failing ~4 times/day

My tests landed around the time this started crashing. Might just be that they're triggering something old.

This patch seems to have upped the repro rate to permorange: https://treeherder.mozilla.org/#/jobs?repo=try&revision=470227546678

(In reply to Andreas Pehrson [:pehrsons] (Telenor) from comment #75) > This patch seems to have upped the repro rate to permorange: > https://treeherder.mozilla.org/#/jobs?repo=try&revision=470227546678 Milan/Bas/Jeff - this patch should make it a lot easier to debug!

Jeff's patch from comment 86 https://treeherder.mozilla.org/#/jobs?repo=try&revision=14f1b0b5402c Benoit, can you follow up?

Comment on attachment 8617340 [details] [diff] [review] Here's a patch that might help debug this Review of attachment 8617340 [details] [diff] [review]: ----------------------------------------------------------------- https://treeherder.mozilla.org/#/jobs?repo=try&revision=14f1b0b5402c

This looks green, we should land it and see if it helps with the intermittents.

Comment on attachment 8617340 [details] [diff] [review] Here's a patch that might help debug this Review of attachment 8617340 [details] [diff] [review]: ----------------------------------------------------------------- ::: gfx/2d/2D.h @@ +388,5 @@ > struct MappedSurface { > uint8_t *mData; > int32_t mStride; > +#ifdef DEBUG > + RefPtr<DataSourceSurface> mSrc; Am I missing some magic or is mSrc never set?

I don't see it set, but there could be magic.

Comment on attachment 8621251 [details] [diff] [review] Indeed Review of attachment 8621251 [details] [diff] [review]: ----------------------------------------------------------------- ::: gfx/2d/2D.h @@ +469,5 @@ > virtual bool Map(MapType, MappedSurface *aMappedSurface) > { > aMappedSurface->mData = GetData(); > aMappedSurface->mStride = Stride(); > + aMappedSurface->mSrc = this; I imagine inside of #ifdef DEBUG

Well, patch worked :-) https://treeherder.mozilla.org/#/jobs?repo=try&revision=81fc7e34c961

(In reply to Andreas Pehrson [:pehrsons] (Telenor) from comment #107) > Well, patch worked :-) > https://treeherder.mozilla.org/#/jobs?repo=try&revision=81fc7e34c961 Or not, well, don't know, but the assert is because of Map() of a surface which previously saw GetData().

Let's see this instead: https://treeherder.mozilla.org/#/jobs?repo=try&revision=8bbea8a47a9e It's based on a recent m-c with plenty of D2D GetData() replaced by Map().

Here, https://treeherder.mozilla.org/#/jobs?repo=try&revision=18cc762afc9c So there's one assertion failure on WinXP, probably unrelated. > 21:05:18 INFO - Assertion failure: !mSrc->mIsMapped, at c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\obj-firefox\dist\include\mozilla/gfx/2D.h:394 > 21:05:31 INFO - #01: mozilla::gfx::GfxPatternToCairoPattern [gfx/2d/DrawTargetCairo.cpp:487] I also applied the patch to the various D2D surfaces, but it didn't yield anything. Seems to be a GetData/Map mix on the Win7 machine though, perhaps that's something.

Looks like the debug patch didn't catch anything. I'm not too familiar with the code here but I have some time. I'll run a manual audit of the Map/GetData and see if I can spot something. Keeping ni?

One of them crashes at SourceSurfaceD2DTarget.cpp:120 mBitmap = nullptr; in d2d1.dll (http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/b2g-inbound-win32/1434459621/b2g-inbound_win7-ix_test-mochitest-1-bm110-tests1-windows-build1.txt.gz)

Looks like we're calling DrawTargetWillChange for a target that's mapped.

Comment on attachment 8623239 [details] Relevant crash stack I'm not sure this is related to this bug actually. Let's make sure we don't mix things up.

Going to focus on this one failure: https://hg.mozilla.org/try/annotate/18cc762afc9c0679acccc932aca6c13ae99c2cc3/gfx/2d/SourceSurfaceD2DTarget.cpp#l296 I'm not sure it's the same bug but fixing that may help here.

There is bug 1133119, but the timing isn't right, and that one would have just increased the number of times we fail the Map call, but thought I'd mention it anyway.

mMapped vs. mIsMapped is annoying. Just sayin'.

This is mostly a shot in the dark but could be worth running through try as well. I don't know how we're getting here: http://mxr.mozilla.org/mozilla-central/source/gfx/2d/DrawTargetD2D.cpp#1232 without having the surface mapped since we map it above and check of errors properly.

In this failure at least - https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=10921382 - one of the non-crashing threads (32) is in the middle of doing this: 00:40:33 INFO - 2 xul.dll!mozilla::layers::TextureHost::DestroyIPDLActor(mozilla::layers::PTextureParent *) [CompositableHost.cpp:a8e0bde30bd4 : 95 + 0xf] 00:40:33 INFO - 3 xul.dll!mozilla::layers::PTextureParent::Send__delete__(mozilla::layers::PTextureParent *) [PTextureParent.cpp:a8e0bde30bd4 : 64 + 0xc]

I still haven't reproduced this yet but I'll give it another try. I did find this handy command: ./mach mochitest --run-until-failure --repeat 300 ./dom/canvas/test/test_capture.html

Benoit, please see bug 1176363 comment 13 for a race between main thread and the compositor that could be related to this bug as well. Ideas for fixes welcome.