ntdll.dll!775a9d11() Unknown [Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll] [External Code] FsDomNodeFirefox.dll!6bae1b93() Unknown FsDomNodeFirefox.dll!6baf0851() Unknown FsDomNodeFirefox.dll!6bb0272d() Unknown FsDomNodeFirefox.dll!6bb02995() Unknown FsDomNodeFirefox.dll!6bb05be3() Unknown FsDomNodeFirefox.dll!6bae789b() Unknown [External Code] FsDomSrv.dll!6bbcead5() Unknown [External Code] FsDomSrv.dll!6bbcf048() Unknown FsDomSrv.dll!6bbcf05a() Unknown FsDomSrv.dll!6bb72f50() Unknown [External Code] jhook.dll!6bd43eb1() Unknown jhook.dll!6bd66356() Unknown jhook.dll!6bd682a2() Unknown [External Code] jhook.dll!6bd684af() Unknown jhook.dll!6bd69c42() Unknown > xul.dll!nsIFrame::BuildDisplayListForChild(nsDisplayListBuilder * aBuilder, nsIFrame * aChild, const nsRect & aDirtyRect, const nsDisplayListSet & aLists, unsigned int aFlags) Line 3079 C++ xul.dll!nsGlobalWindow::QueryInterface(const nsID & aIID, void * * aInstancePtr) Line 2082 C++ xul.dll!nsGlobalChromeWindow::QueryInterface(const nsID & aIID, void * * aInstancePtr) Line 13667 C++ [External Code] xul.dll!nsDocShell::GetPresShell() Line 1876 C++ xul.dll!nsGlobalWindow::GetInnerScreenRect() Line 5867 C++ xul.dll!JS::CanonicalizeNaN(double d) Line 254 C++ xul.dll!mozilla::dom::WindowBinding::get_mozInnerScreenY(JSContext * cx, JS::Handle<JSObject *> obj, nsGlobalWindow * self, JSJitGetterCallArgs args) Line 4808 C++ xul.dll!mozilla::dom::WindowBinding::genericGetter(JSContext * cx, unsigned int argc, JS::Value * vp) Line 15769 C++ [External Code] xul.dll!nsIFrame::GetBorderRadii(int * aRadii) Line 1452 C++ xul.dll!RoundedBorderIntersectsRect(nsIFrame * aFrame, const nsPoint & aFrameToReferenceFrame, const nsRect & aTestRect) Line 3212 C++ xul.dll!nsTArray_Impl<FramesWithDepth,nsTArrayInfallibleAllocator>::RemoveElementsAt(unsigned int aStart, unsigned int aCount) Line 1987 C++ xul.dll!nsTArray_base<nsTArrayFallibleAllocator,nsTArray_CopyWithConstructors<mozilla::dom::ClonedMessageData> >::~nsTArray_base<nsTArrayFallibleAllocator,nsTArray_CopyWithConstructors<mozilla::dom::ClonedMessageData> >() Line 21 C++ FsDomSrv.dll!6bb73ce2() Unknown [External Code] xul.dll!mozilla::a11y::AccessibleWrap::FireWinEvent(mozilla::a11y::Accessible * aTarget, unsigned int aEventType) Line 1153 C++ xul.dll!mozilla::a11y::ProxyTextChangeEvent(mozilla::a11y::ProxyAccessible * aText, const nsString & aStr, int aStart, unsigned int aLen, bool aInsert, bool __formal) Line 139 C++ xul.dll!mozilla::a11y::DocAccessibleParent::RecvTextChangeEvent(const unsigned __int64 & aID, const nsString & aStr, const int & aStart, const unsigned int & aLen, const bool & aIsInsert, const bool & aFromUser) Line 322 C++ xul.dll!mozilla::a11y::PDocAccessibleParent::OnMessageReceived(const IPC::Message & msg__) Line 416 C++ xul.dll!mozilla::dom::PContentParent::OnMessageReceived(const IPC::Message & msg__) Line 2977 C++ xul.dll!mozilla::ipc::MessageChannel::DispatchAsyncMessage(const IPC::Message & aMsg) Line 1796 C++ xul.dll!mozilla::ipc::MessageChannel::DispatchMessageW(IPC::Message && aMsg) Line 1733 C++ xul.dll!mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask & aTask) Line 1603 C++ xul.dll!mozilla::ipc::MessageChannel::MessageTask::Run() Line 1637 C++ xul.dll!nsThread::ProcessNextEvent(bool aMayWait, bool * aResult) Line 1265 C++ xul.dll!NS_ProcessNextEvent(nsIThread * aThread, bool aMayWait) Line 389 C++ xul.dll!mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate * aDelegate) Line 96 C++ xul.dll!MessageLoop::RunHandler() Line 232 C++ xul.dll!MessageLoop::Run() Line 212 C++ xul.dll!nsBaseAppShell::Run() Line 158 C++ xul.dll!nsAppShell::Run() Line 265 C++ xul.dll!nsAppStartup::Run() Line 284 C++ xul.dll!XREMain::XRE_mainRun() Line 4476 C++ xul.dll!XREMain::XRE_main(int argc, char * * argv, const mozilla::BootstrapConfig & aConfig) Line 4654 C++ xul.dll!XRE_main(int argc, char * * argv, const mozilla::BootstrapConfig & aConfig) Line 4745 C++ xul.dll!mozilla::BootstrapImpl::XRE_main(int argc, char * * argv, const mozilla::BootstrapConfig & aConfig) Line 45 C++ firefox.exe!do_main(int argc, char * * argv, char * * envp) Line 237 C++ firefox.exe!NS_internal_main(int argc, char * * argv, char * * envp) Line 309 C++ firefox.exe!wmain(int argc, wchar_t * * argv) Line 118 C++ [External Code]

Another stack: Exception thrown at 0x774F6DC9 (ntdll.dll) in firefox.exe: 0xC0000005: Access violation writing location 0x00000014. ntdll.dll!774f6dc9() Unknown [Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll] [External Code] FsDomSrv.dll!658efdc9() Unknown FsDomSrv.dll!658f3639() Unknown FsDomSrv.dll!658d6858() Unknown [External Code] > xul.dll!mozilla::ipc::MessageChannel::ExitedCxxStack() Line 1921 C++ xul.dll!mozilla::ipc::MessageChannel::CxxStackFrame::~CxxStackFrame() Line 289 C++ xul.dll!mozilla::ipc::MessageChannel::Send(IPC::Message * aMsg) Line 803 C++ xul.dll!mozilla::dom::PBrowserParent::SendRealMouseMoveEvent(const mozilla::WidgetMouseEvent & event, const mozilla::layers::ScrollableLayerGuid & aGuid, const unsigned __int64 & aInputBlockId) Line 581 C++ xul.dll!mozilla::dom::TabParent::SendRealMouseEvent(mozilla::WidgetMouseEvent & aEvent) Line 1147 C++ [External Code] jhook.dll!665e3e57() Unknown [External Code] xul.dll!nsBaseAppShell::DoProcessNextNativeEvent(bool mayWait) Line 139 C++ xul.dll!nsBaseAppShell::OnProcessNextEvent(nsIThreadInternal * thr, bool mayWait) Line 289 C++ xul.dll!nsThread::ProcessNextEvent(bool aMayWait, bool * aResult) Line 1223 C++ xul.dll!NS_ProcessNextEvent(nsIThread * aThread, bool aMayWait) Line 389 C++ xul.dll!mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate * aDelegate) Line 124 C++ xul.dll!MessageLoop::RunHandler() Line 232 C++ xul.dll!MessageLoop::Run() Line 212 C++ xul.dll!nsBaseAppShell::Run() Line 158 C++ xul.dll!nsAppShell::Run() Line 265 C++ xul.dll!nsAppStartup::Run() Line 284 C++ xul.dll!XREMain::XRE_mainRun() Line 4476 C++ xul.dll!XREMain::XRE_main(int argc, char * * argv, const mozilla::BootstrapConfig & aConfig) Line 4654 C++ xul.dll!XRE_main(int argc, char * * argv, const mozilla::BootstrapConfig & aConfig) Line 4745 C++ xul.dll!mozilla::BootstrapImpl::XRE_main(int argc, char * * argv, const mozilla::BootstrapConfig & aConfig) Line 45 C++ firefox.exe!do_main(int argc, char * * argv, char * * envp) Line 237 C++ firefox.exe!NS_internal_main(int argc, char * * argv, char * * envp) Line 309 C++ firefox.exe!wmain(int argc, wchar_t * * argv) Line 118 C++ [External Code]

So, a couple things. First why is there no symbols for ntdll.dll? are you not using the microsoft symbol / source servers? After that might be interesting to disassemble around the crashing eip to see if the code looks somewhat reasonable or if you jumped into something random. Worth seeing what the stack with symbols is before doing this though. Third maybe try a build using cfguard? try getting a better stack first, and I'm not sure where the /cfguard bug is at, should probably check that.

here is a better stack: ntdll.dll!_RtlReportCriticalFailure@12() Unknown ntdll.dll!_RtlpReportHeapFailure@4() Unknown ntdll.dll!_RtlpHeapHandleError@4() Unknown ntdll.dll!_RtlpLogHeapFailure@24() Unknown ntdll.dll!_RtlFreeHeap@12() Unknown ucrtbase.dll!_free_base() Unknown ucrtbase.dll!_free() Unknown FsDomNodeFirefox.dll!52bf1b93() Unknown [Frames below may be incorrect and/or missing, no symbols loaded for FsDomNodeFirefox.dll] FsDomNodeFirefox.dll!52c00851() Unknown FsDomNodeFirefox.dll!52c1272d() Unknown FsDomNodeFirefox.dll!52c12995() Unknown FsDomNodeFirefox.dll!52c15be3() Unknown FsDomNodeFirefox.dll!52bf789b() Unknown [External Code] FsDomSrv.dll!52cdead5() Unknown [External Code] FsDomSrv.dll!52cdf048() Unknown FsDomSrv.dll!52cdf05a() Unknown FsDomSrv.dll!52c82f50() Unknown [External Code] jhook.dll!589a3eb1() Unknown jhook.dll!589c6356() Unknown > xul.dll!NS_TableDrivenQI(void * aThis, const nsID & aIID, void * * aInstancePtr, const QITableEntry * aEntries) Line 18 C++ [External Code] xul.dll!PLDHashTable::SearchTable<1>(const void * aKey, unsigned int aKeyHash) Line 388 C++ xul.dll!mozilla::net::nsStandardURL::QueryInterface(const nsID & aIID, void * * aInstancePtr) Line 1249 C++

(In reply to Yura Zenevich [:yzen] from comment #3) > here is a better stack: > > <snip> That one is clearly heap corruption and should be treated as a separate bug.

(In reply to Aaron Klotz [:aklotz] from comment #4) > That one is clearly heap corruption and should be treated as a separate bug. Sorry, I'm thinking of that other bug. Yes, this is heap corruption. But it is in the Windows heap, not the jemalloc heap, so it can't be directly caused by heap activity in our code. Most likely a problem inside JAWS.

(In reply to Aaron Klotz [:aklotz] from comment #5) > so it can't be directly caused by heap activity in our code. I say "directly" because, of course, because any Windows APIs will use the Windows heap, not jemalloc. But I find it highly unlikely that any Windows APIs that we call would corrupt the heap.

Asked JAWS developer about the crash stack, here's the response (Aaron, Trevor, you might have some ideas?): W/R/T the crash: I definitely see a crash in our FSDomNodeFirefox.dll. Specifically, this happens with hide and show events. We hold on to a document root which seems to be going away part of the way through the hide/show event processing. We obtain the root node that is going away by doing the Following: static CComPtr<IAccessible> GetContainerIAccessible(IAccessible* acc) { if (!acc) return nullptr; CComQIPtr<IServiceProvider> serviceProvider = acc; if (!serviceProvider) return nullptr; CComPtr<IAccessible>rootDocument; HRESULT hr = serviceProvider->QueryService(SID_IAccessibleContentDocument, IID_IAccessible, (void**)&rootDocument); if (hr != S_OK) return nullptr; return rootDocument; } We seem to get several events on a document with the name of the page but without any children. I am not sure exactly what event is triggering this. Immediately after we try building the document without any children, I see a number of exceptions like: Exception thrown at 0x74513112 (KernelBase.dll) in firefox.exe: 0x80020008: Bad variable type. Exception thrown at 0x74513112 (KernelBase.dll) in firefox.exe: 0x80020008: Bad variable type.

(In reply to Yura Zenevich [:yzen] from comment #7) > Asked JAWS developer about the crash stack, here's the response (Aaron, > Trevor, you might have some ideas?): > > W/R/T the crash: > > I definitely see a crash in our FSDomNodeFirefox.dll. > > Specifically, this happens with hide and show events. We hold on to a > document root which seems to be going away part of the way through the > hide/show event processing. > > We obtain the root node that is going away by doing the > > Following: I'm not really sure what this has to do with the crash, I guess this is the crashing code? > > static CComPtr<IAccessible> GetContainerIAccessible(IAccessible* acc) > > { > > if (!acc) return nullptr; > > > > CComQIPtr<IServiceProvider> serviceProvider = acc; > > if (!serviceProvider) return nullptr; > > > > CComPtr<IAccessible>rootDocument; > > HRESULT hr = > serviceProvider->QueryService(SID_IAccessibleContentDocument, > IID_IAccessible, (void**)&rootDocument); that just using & seems sketchy, but it seems to be normal for ComPtr which I know basically nothing about. > > if (hr != S_OK) return nullptr; > > > > return rootDocument; > > } seems rather odd this code is doing any allocating or freeing, but maybe I'm missing something. It seems pretty clear from the last stack we're either freeing something invalid or the heap was already corrupted. > I am not sure exactly what event is triggering this. events seem sort of unrelated unless we are talking about different things. The question seems to be when was the heap first corrupted and by who. > Immediately after we try building the document without any children, I see a > number of exceptions like: > > Exception thrown at 0x74513112 (KernelBase.dll) in firefox.exe: 0x80020008: > Bad variable type. > > Exception thrown at 0x74513112 (KernelBase.dll) in firefox.exe: 0x80020008: > Bad variable type. no clue what that means without poking around at windows stuff for those strings. Maybe it means errors in COM proxy stuff with bad marshaling? Maybe it means the heap is bad I'm not sure where it comes from.

I think there might be a problem with the way that we wrap the outparam in IServiceProvider::QueryService: it looks to me like it's being wrapped as an IUnknown (even though we know the true IID; the COM interceptor is not recognizing that fact). So we're proxying an IUnknown but the client (as you can see in the sample code) assumes that it has an IAccessible. That *might* be why we're seeing funny memory stuff here. I'll file a bug for that and have it block this one.

Still seeing this crash stack with trunk build: > ntdll.dll!_RtlReportCriticalFailure@12() Unknown ntdll.dll!_RtlpReportHeapFailure@4() Unknown ntdll.dll!_RtlpHeapHandleError@4() Unknown ntdll.dll!_RtlpLogHeapFailure@24() Unknown ntdll.dll!_RtlFreeHeap@12() Unknown ucrtbase.dll!_free_base() Unknown ucrtbase.dll!_free() Unknown FsDomNodeFirefox.dll!65721b93() Unknown [Frames below may be incorrect and/or missing, no symbols loaded for FsDomNodeFirefox.dll] FsDomNodeFirefox.dll!65730851() Unknown FsDomNodeFirefox.dll!6574272d() Unknown FsDomNodeFirefox.dll!65742995() Unknown FsDomNodeFirefox.dll!65745be3() Unknown FsDomNodeFirefox.dll!6572789b() Unknown [External Code] FsDomSrv.dll!661bead5() Unknown [External Code] FsDomSrv.dll!661bf048() Unknown FsDomSrv.dll!661bf05a() Unknown FsDomSrv.dll!66162f50() Unknown [External Code]

Just to confirm (haven't tried GFlags yet) this I can get a crash fairly easilly (just go to reddit.com) though the signature is a bit different: Unhandled exception at 0x777B9D11 (ntdll.dll) in firefox.exe: 0xC0000374: A heap has been corrupted (parameters: 0x777ED8D0). ntdll.dll!_RtlReportCriticalFailure@12() Unknown ntdll.dll!_RtlpReportHeapFailure@4() Unknown ntdll.dll!_RtlpHeapHandleError@4() Unknown ntdll.dll!_RtlpLogHeapFailure@24() Unknown ntdll.dll!_RtlFreeHeap@12() Unknown ucrtbase.dll!_free_base() Unknown ucrtbase.dll!_free() Unknown FsDomNodeFirefox.dll!6af91b93() Unknown [Frames below may be incorrect and/or missing, no symbols loaded for FsDomNodeFirefox.dll] FsDomNodeFirefox.dll!6afa0851() Unknown FsDomNodeFirefox.dll!6afb272d() Unknown FsDomNodeFirefox.dll!6afb2995() Unknown FsDomNodeFirefox.dll!6afb5be3() Unknown FsDomNodeFirefox.dll!6af9789b() Unknown [External Code] FsDomSrv.dll!6b07ead5() Unknown [External Code] FsDomSrv.dll!6b07f048() Unknown FsDomSrv.dll!6b07f05a() Unknown FsDomSrv.dll!6b022f50() Unknown [External Code] > xul.dll!mozilla::net::nsStandardURL::QueryInterface(const nsID & aIID, void * * aInstancePtr) Line 1252 C++ [External Code] xul.dll!mozilla::net::nsPACMan::AsyncGetProxyForURI(nsIURI * uri, mozilla::net::nsPACManCallback * callback, bool mainThreadResponse) Line 364 C++

If I go to twitter.com I get exactly the same stack as in comment 10

I can reproduce a slightly better stack. Heap diagnostics are telling me that it's a double-free. I'll try running UMDH to see if I can figure out where its corresponding allocation came from.

This has been fixed by JAWS.