Filed by: cbook [at] mozilla.com https://treeherder.mozilla.org/logviewer.html#?job_id=83032652&repo=mozilla-inbound https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-inbound-win32-debug/1489144882/mozilla-inbound_win7_vm-debug_test-marionette-e10s-bm138-tests1-windows-build1047.txt.gz

this started Friday March 10th, and has 33 failures since then, I am going to work on getting more information (retriggers, log data)

some retriggers here: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=mn-e10s%20win%20debug&tochange=68e6acf47dabf0c1aff5e3b8ab246a4bce63d645&fromchange=560eb8e7d89fcb8d51642dd6b02087c75b119e3e here is a clip from a win8 log [0] : 04:36:41 INFO - TEST-START | test_navigation.py TestBackForwardNavigation.test_frameset_after_navigating_in_frame 04:36:48 INFO - mozcrash Downloading symbols from: https://queue.taskcluster.net/v1/task/KRClRAR4RmK6ZMmHYal_Gg/artifacts/public/build/firefox-55.0a1.en-US.win64.crashreporter-symbols.zip 04:36:56 INFO - mozcrash Copy/paste: C:\slave\test\build\win32-minidump_stackwalk.exe c:\users\cltbld~1.t-w\appdata\local\temp\tmpl2huju.mozrunner\minidumps\b06e7d66-afd0-4292-914f-5ee2595d02d3.dmp c:\users\cltbld~1.t-w\appdata\local\temp\tmpabpnzh 04:37:03 INFO - mozcrash Saved minidump as C:\slave\test\build\blobber_upload_dir\b06e7d66-afd0-4292-914f-5ee2595d02d3.dmp 04:37:03 INFO - mozcrash Saved app info as C:\slave\test\build\blobber_upload_dir\b06e7d66-afd0-4292-914f-5ee2595d02d3.extra 04:37:03 ERROR - PROCESS-CRASH | test_navigation.py TestBackForwardNavigation.test_frameset_after_navigating_in_frame | application crashed [@ mozilla::a11y::ProxyDestroyed(mozilla::a11y::ProxyAccessible *)] 04:37:03 INFO - Crash dump filename: c:\users\cltbld~1.t-w\appdata\local\temp\tmpl2huju.mozrunner\minidumps\b06e7d66-afd0-4292-914f-5ee2595d02d3.dmp 04:37:03 INFO - Operating system: Windows NT 04:37:03 INFO - 6.2.9200 04:37:03 INFO - CPU: amd64 04:37:03 INFO - family 6 model 30 stepping 5 04:37:03 INFO - 8 CPUs 04:37:03 INFO - GPU: UNKNOWN 04:37:03 INFO - Crash reason: EXCEPTION_BREAKPOINT 04:37:03 INFO - Crash address: 0x7fd4fcad3e4 04:37:03 INFO - Process uptime: 242 seconds 04:37:03 INFO - Thread 0 (crashed) 04:37:03 INFO - 0 xul.dll!mozilla::a11y::ProxyDestroyed(mozilla::a11y::ProxyAccessible *) [Platform.cpp:68e6acf47dab : 89 + 0x1b] 04:37:03 INFO - rax = 0x0000000000000000 rdx = 0x0000009339f8ef38 04:37:03 INFO - rcx = 0x00000000ffffffff rbx = 0x0000000000000059 04:37:03 INFO - rsi = 0x0000000000000137 rdi = 0x0000000000000000 04:37:03 INFO - rbp = 0x0000009339f8efa8 rsp = 0x0000009339f8ef20 04:37:03 INFO - r8 = 0x0000009339f8ef30 r9 = 0x0000009339f8ef28 04:37:03 INFO - r10 = 0x0000000000000000 r11 = 0x0000009339f8a470 04:37:03 INFO - r12 = 0x0000009350e84088 r13 = 0x0000000000000004 04:37:03 INFO - r14 = 0x0000009350e84000 r15 = 0x000000000000000f 04:37:03 INFO - rip = 0x000007fd4fcad3e4 04:37:03 INFO - Found by: given as instruction pointer in context 04:37:03 INFO - 1 xul.dll!mozilla::a11y::DocAccessibleParent::Destroy() [DocAccessibleParent.cpp:68e6acf47dab : 522 + 0x9] 04:37:03 INFO - rbx = 0x0000000000000059 rbp = 0x0000009339f8efa8 04:37:03 INFO - rsp = 0x0000009339f8ef50 r12 = 0x0000009350e84088 04:37:03 INFO - r13 = 0x0000000000000004 r14 = 0x0000009350e84000 04:37:03 INFO - r15 = 0x000000000000000f rip = 0x000007fd4fd14f91 04:37:03 INFO - Found by: call frame info 04:37:03 INFO - 2 xul.dll!mozilla::a11y::DocAccessibleParent::ActorDestroy(mozilla::ipc::IProtocol::ActorDestroyReason) [DocAccessibleParent.h:68e6acf47dab : 114 + 0x8] 04:37:03 INFO - rbx = 0x0000000000000059 rbp = 0x0000009339f8efa8 04:37:03 INFO - rsp = 0x0000009339f8efd0 r12 = 0x0000009350e84088 04:37:03 INFO - r13 = 0x0000000000000004 r14 = 0x0000009350e84000 04:37:03 INFO - r15 = 0x000000000000000f rip = 0x000007fd4eb0a9c7 04:37:03 INFO - Found by: call frame info 04:37:03 INFO - 3 xul.dll!mozilla::dom::PBrowserParent::DestroySubtree(mozilla::ipc::IProtocol::ActorDestroyReason) [PBrowserParent.cpp:68e6acf47dab : 4966 + 0xc] 04:37:03 INFO - rbx = 0x0000000000000059 rbp = 0x0000009339f8efa8 04:37:03 INFO - rsp = 0x0000009339f8f000 r12 = 0x0000009350e84088 04:37:03 INFO - r13 = 0x0000000000000004 r14 = 0x0000009350e84000 04:37:03 INFO - r15 = 0x000000000000000f rip = 0x000007fd4ce2f38c 04:37:03 INFO - Found by: call frame info 04:37:03 INFO - 4 xul.dll!mozilla::dom::PContentParent::DestroySubtree(mozilla::ipc::IProtocol::ActorDestroyReason) [PContentParent.cpp:68e6acf47dab : 8610 + 0xc] 04:37:03 INFO - rbx = 0x0000000000000059 rbp = 0x0000009339f8efa8 04:37:03 INFO - rsp = 0x0000009339f8f050 r12 = 0x0000009350e84088 04:37:03 INFO - r13 = 0x0000000000000004 r14 = 0x0000009350e84000 04:37:03 INFO - r15 = 0x000000000000000f rip = 0x000007fd4cea66cb 04:37:03 INFO - Found by: call frame info 04:37:03 INFO - 5 xul.dll!mozilla::dom::PContentParent::OnChannelError() [PContentParent.cpp:68e6acf47dab : 8575 + 0xd] 04:37:03 INFO - rbx = 0x0000000000000059 rbp = 0x0000009339f8efa8 04:37:03 INFO - rsp = 0x0000009339f8f0a0 r12 = 0x0000009350e84088 04:37:03 INFO - r13 = 0x0000000000000004 r14 = 0x0000009350e84000 04:37:03 INFO - r15 = 0x000000000000000f rip = 0x000007fd4ceabdbf 04:37:03 INFO - Found by: call frame info 04:37:03 INFO - 6 xul.dll!mozilla::dom::ContentParent::OnChannelError() [ContentParent.cpp:68e6acf47dab : 1527 + 0x8] 04:37:03 INFO - rbx = 0x0000000000000059 rbp = 0x0000009339f8efa8 04:37:03 INFO - rsp = 0x0000009339f8f0d0 r12 = 0x0000009350e84088 04:37:03 INFO - r13 = 0x0000000000000004 r14 = 0x0000009350e84000 04:37:03 INFO - r15 = 0x000000000000000f rip = 0x000007fd4eaf2272 04:37:03 INFO - Found by: call frame info 04:37:03 INFO - 7 xul.dll!mozilla::ipc::MessageChannel::NotifyMaybeChannelError() [MessageChannel.cpp:68e6acf47dab : 2238 + 0x14] 04:37:03 INFO - rbx = 0x0000000000000059 rbp = 0x0000009339f8efa8 04:37:03 INFO - rsp = 0x0000009339f8f100 r12 = 0x0000009350e84088 04:37:03 INFO - r13 = 0x0000000000000004 r14 = 0x0000009350e84000 04:37:03 INFO - r15 = 0x000000000000000f rip = 0x000007fd4cb487e9 04:37:03 INFO - Found by: call frame info 04:37:03 INFO - 8 xul.dll!mozilla::ipc::MessageChannel::OnNotifyMaybeChannelError() [MessageChannel.cpp:68e6acf47dab : 2267 + 0x8] 04:37:03 INFO - rbx = 0x0000000000000059 rbp = 0x0000009339f8efa8 04:37:03 INFO - rsp = 0x0000009339f8f130 r12 = 0x0000009350e84088 04:37:03 INFO - r13 = 0x0000000000000004 r14 = 0x0000009350e84000 04:37:03 INFO - r15 = 0x000000000000000f rip = 0x000007fd4cb49c62 04:37:03 INFO - Found by: call frame info 04:37:03 INFO - 9 xul.dll!mozilla::detail::RunnableMethodImpl<mozilla::ipc::MessageChannel * const,void ( mozilla::ipc::MessageChannel::*)(void),0,1>::Run() [nsThreadUtils.h:68e6acf47dab : 890 + 0x3] 04:37:03 INFO - rbx = 0x0000000000000059 rbp = 0x0000009339f8efa8 04:37:03 INFO - rsp = 0x0000009339f8f160 r12 = 0x0000009350e84088 04:37:03 INFO - r13 = 0x0000000000000004 r14 = 0x0000009350e84000 04:37:03 INFO - r15 = 0x000000000000000f rip = 0x000007fd4cb4dd23 04:37:03 INFO - Found by: call frame info 04:37:03 INFO - 10 xul.dll!nsThread::ProcessNextEvent(bool,bool *) [nsThread.cpp:68e6acf47dab : 1264 + 0x14] 04:37:03 INFO - rbx = 0x0000000000000059 rbp = 0x0000009339f8efa8 04:37:03 INFO - rsp = 0x0000009339f8f190 r12 = 0x0000009350e84088 04:37:03 INFO - r13 = 0x0000000000000004 r14 = 0x0000009350e84000 04:37:03 INFO - r15 = 0x000000000000000f rip = 0x000007fd4c650027 04:37:03 INFO - Found by: call frame info [0] https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-inbound-win64-debug/1489144882/mozilla-inbound_win8_64-debug_test-marionette-e10s-bm111-tests1-windows-build301.txt.gz I am not sure what other information to provide here- I will follow up when I have a smaller range for the breakage. :gijs, I see you are the triage owner here, if this is not your area of code or test source, please let me know- otherwise, can you find someone on the team to take a look at this. I suspect this will fail at a steady rate to keep on our radar

:surkov or :tbsaunde would be better bets for tracking this down given this is core code crashing (rather than frontend not adding the right attributes to make something accessible, or something like that). A regression window would likely help. Do we have (even a broad) one?

I believe this set of patches is the range: https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=af8309c5eeba although I am not sure why they would be problematic. here are my retriggers: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=mn-e10s%20win%20debug&tochange=68e6acf47dabf0c1aff5e3b8ab246a4bce63d645&fromchange=560eb8e7d89fcb8d51642dd6b02087c75b119e3e&selectedJob=83802069

:tbsaunde, can you take at this intermittent- ideally fix it, but at least help narrow down where this is happening?

yeah, I'm looking.

(In reply to Joel Maher ( :jmaher) from comment #5) > I believe this set of patches is the range: > https://hg.mozilla.org/integration/mozilla-inbound/ > pushloghtml?changeset=af8309c5eeba > > although I am not sure why they would be problematic. > > > here are my retriggers: > https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter- > searchStr=mn- > e10s%20win%20debug&tochange=68e6acf47dabf0c1aff5e3b8ab246a4bce63d645&fromchan > ge=560eb8e7d89fcb8d51642dd6b02087c75b119e3e&selectedJob=83802069 test_navigation.py itself was updated in bug 1330348 around this time.

somehow this has greatly reduced in frequency.

And it doesn't happen anymore. So something should have fixed this bug. Given it is unknown I will mark it as WFM.