User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:36.0) Gecko/20100101 Firefox/36.0 SeaMonkey/2.33.1 Build ID: 20150321194901 Steps to reproduce: When SeaMonkey decides to compact folders (to save storage space), the process runs for 10-20 seconds, works for a few folders, then crashes, leaving the current account in a corrupt state (clicking a message shows the body of a different message). This might be a dupe of bug 1066998, but I have a different crash signature. My mailnews setup is moderately complex: 8 IMAP accounts (TLS IMAP port 993) 2 POP3 accounts (TLS POP3 port 995) 2 NNTP accounts (clear text port 119) 1 NNTP account (TLS NNTP port 563)

Crash Report [@ nsCOMPtr_base::assign_with_AddRef(nsISupports*) | nsImapMockChannel::Close() ] bp-b4894bd5-cdf0-4ba2-8cc8-cf9f62150426 bp-51cb6fae-063f-42c9-a385-ba8092150214 bp-10b03a26-38e9-4288-aa13-0795b2141025 bp-eeec13b8-7f96-458e-a2f7-ba9da2140919 nsCOMPtr_base::assign_with_AddRef(nsISupports*) nsImapMockChannel::Close() nsImapCacheStreamListener::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsInputStreamPump::OnStateStop() nsInputStreamPump::OnInputStreamReady(nsIAsyncInputStream*) nsInputStreamReadyEvent::Run() nsThread::ProcessNextEvent(bool, bool*) NS_ProcessNextEvent(nsIThread*, bool) nsThread::Shutdown() nsRunnableMethodImpl<void ( mozilla::dom::DOMStorageCacheBridge::*)(void), void, 0>::Run() nsThread::ProcessNextEvent(bool, bool*) NS_ProcessNextEvent(nsIThread*, bool) nsThread::Shutdown() nsRunnableMethodImpl<void ( mozilla::dom::DOMStorageCacheBridge::*)(void), void, 0>::Run() nsThread::ProcessNextEvent(bool, bool*) NS_ProcessNextEvent(nsIThread*, bool) mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) MessageLoop::RunHandler() MessageLoop::Run() nsBaseAppShell::Run() nsAppShell::Run() nsAppStartup::Run() XREMain::XRE_mainRun() XREMain::XRE_main(int, char** const, nsXREAppData const*) XRE_main do_main NS_internal_main(int, char**) wmain __tmainCRTStartup BaseThreadInitThunk __RtlUserThreadStart _RtlUserThreadStart

I have another crash signature, might be related. Crash Report [@ nsCOMPtr_base::assign_with_AddRef(nsISupports*) | nsImapCacheStreamListener::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) ] bp-277399c2-b5c2-4fb7-8d24-3851d2150426 This seems to be the same signature from bug 1066998 nsCOMPtr_base::assign_with_AddRef(nsISupports*) nsImapCacheStreamListener::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsInputStreamPump::OnStateStop() nsInputStreamPump::OnInputStreamReady(nsIAsyncInputStream*) nsOutputStreamReadyEvent::Run() nsThread::ProcessNextEvent(bool, bool*) NS_ProcessNextEvent(nsIThread*, bool) nsThread::Shutdown() nsRunnableMethodImpl<tag_nsresult ( nsIThread::*)(void), void, 1>::Run() nsThread::ProcessNextEvent(bool, bool*) NS_ProcessNextEvent(nsIThread*, bool) nsThread::Shutdown() nsRunnableMethodImpl<tag_nsresult ( nsIThread::*)(void), void, 1>::Run() nsThread::ProcessNextEvent(bool, bool*) NS_ProcessNextEvent(nsIThread*, bool) nsThread::Shutdown() nsRunnableMethodImpl<tag_nsresult ( nsIThread::*)(void), void, 1>::Run() nsThread::ProcessNextEvent(bool, bool*) NS_ProcessNextEvent(nsIThread*, bool) nsThread::Shutdown() nsRunnableMethodImpl<tag_nsresult ( nsIThread::*)(void), void, 1>::Run() nsThread::ProcessNextEvent(bool, bool*) NS_ProcessNextEvent(nsIThread*, bool) mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) MessageLoop::RunHandler() MessageLoop::Run() nsBaseAppShell::Run() nsAppShell::Run() nsAppStartup::Run() XREMain::XRE_mainRun() XREMain::XRE_main(int, char** const, nsXREAppData const*) XRE_main do_main NS_internal_main(int, char**) wmain __tmainCRTStartup BaseThreadInitThunk __RtlUserThreadStart _RtlUserThreadStart

In both stacks, it seems we crash in nsCOMPtr_base::assign_with_AddRef(nsISupports*)

Manually requesting "Compact folders" on one of my IMAP account. Crash Report [@ nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) ] bp-b55add11-aac4-400e-bb85-ae47a2150426 Back trace is 118 functions deep! nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) nsImapMockChannel::ReadFromImapConnection() nsImapMockChannel::AsyncOpen(nsIStreamListener*, nsISupports*) nsImapService::GetMessageFromUrl(nsIImapUrl*, int, nsIMsgFolder*, nsIImapMessageSink*, nsIMsgWindow*, nsISupports*, bool, nsIURI**) nsImapService::StreamMessage(char const*, nsISupports*, nsIMsgWindow*, nsIUrlListener*, bool, nsACString_internal const&, bool, nsIURI**) nsOfflineStoreCompactState::CopyNextMessage(bool&) XREMain::XRE_mainRun() XREMain::XRE_main(int, char** const, nsXREAppData const*) XRE_main do_main NS_internal_main(int, char**) wmain __tmainCRTStartup BaseThreadInitThunk __RtlUserThreadStart _RtlUserThreadStart

One more for the road, with an even deeper stack (128 deep!) [@ nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) ] bp-286da79d-e41a-4743-8ad4-bfe842150426 Crash Address: 0x5a5a5a5a Trying to use a function pointer in a freed struct?

browser.cache.use_new_backend comes with the following comment: Preference for switching the cache backend, can be changed freely at runtime 0 - use the old (Darin's) cache 1 - use the new cache back-end (cache v2) In SM 2.33, default is 0; I changed it to 1. I was then able to compact my 8 IMAP accounts without crashing. I hear that Thunderbird 38 going to fix the problem. How?

(In reply to Mason from comment #6) > browser.cache.use_new_backend comes with the following comment: > > Preference for switching the cache backend, can be changed freely at runtime > 0 - use the old (Darin's) cache > 1 - use the new cache back-end (cache v2) > > In SM 2.33, default is 0; I changed it to 1. > > I was then able to compact my 8 IMAP accounts without crashing. > > I hear that Thunderbird 38 going to fix the problem. How? Probably by switching to the new cache backend.

Mason, thank you for the detailed analysis. CC rkent and jcranmer Hope the mailnews backend experts can identify the cause.

You're welcome. As I stated in comment #1, this bug might be a dupe of bug 1066998 And setting browser.cache.use_new_backend to 1 seems to work around the problem.

Uh-oh, I got a crash even with browser.cache.use_new_backend set to 1. (Manually compacting my main IMAP account) bp-0c5493cc-eab4-4914-926c-2c1d32150506

Two more crashes, one with use_new_backend=1 bp-6e0912d8-d61e-4633-9234-c64732150508 the second with use_new_backend=0 bp-4ea81517-53ce-4a38-8d3d-641ce2150508 Please note that the crash signatures are different! Does the full state of about:config vars appear in crash reports?

bp-6e0912d8-d61e-4633-9234-c64732150508 nsCOMPtr_base::assign_with_AddRef(nsISupports*) | nsImapCacheStreamListener::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) More Reports Search 0 xul.dll nsCOMPtr_base::assign_with_AddRef(nsISupports*) xpcom/glue/nsCOMPtr.cpp 1 xul.dll nsImapCacheStreamListener::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) e:/builds/slave/rel-c-rel-w32-bld/build/mailnews/imap/src/nsImapProtocol.cpp:8678 2 xul.dll nsInputStreamPump::OnStateStop() netwerk/base/src/nsInputStreamPump.cpp bp-4ea81517-53ce-4a38-8d3d-641ce2150508 [@ nsCOMPtr_base::assign_with_AddRef(nsISupports*) | nsImapMockChannel::Close() ] 0 xul.dll nsCOMPtr_base::assign_with_AddRef(nsISupports*) xpcom/glue/nsCOMPtr.cpp 1 xul.dll nsImapMockChannel::Close() e:/builds/slave/rel-c-rel-w32-bld/build/mailnews/imap/src/nsImapProtocol.cpp:8763 2 xul.dll nsImapCacheStreamListener::OnStopRequest(nsIRequest*, nsISupports*, tag_nsresult) e:/builds/slave/rel-c-rel-w32-bld/build/mailnews/imap/src/nsImapProtocol.cpp:8679 3 xul.dll nsInputStreamPump::OnStateStop() netwerk/base/src/nsInputStreamPump.cpp let's focus in bug 1066998

Mason, are you still crashing when using a newer version?

I have a few crashes, but they weren't submitted to Socorro. 0c557e2e-7778-4953-b3c6-0ab5a567fed3 04/02/2016 12:19 1065ab20-a2cc-4fca-8ac3-b29f133258d0 10/01/2016 09:20 a1e67aad-c63d-46f1-af1b-bfa226466530 31/12/2015 11:00 607bdd74-b40a-4bc8-8c32-5afe5b40c91b 20/11/2015 00:25 53a393d1-26cc-430a-83d1-1675a96cba06 20/11/2015 00:17 3e0ca5cd-ef7b-487c-ace7-6b0c374756f9 20/11/2015 00:10 f10f133e-c402-43d9-9247-8d916972a79c 20/10/2015 23:16 91052d26-07ea-45ec-9c19-87a12541581b 20/10/2015 21:32 a73006f9-2b82-4a18-8e0f-8b60d4644472 11/10/2015 17:11 Is there a way to force-push them to the crash server?

(In reply to Mason from comment #14) > I have a few crashes, but they weren't submitted to Socorro. > Is there a way to force-push them to the crash server? I was middle-clicking the crash reports, but only left-clicking works. They are all Flash-related. So the last time I had a crash compacting IMAP folders was August 2015 in SM 2.33.1 bp-16d42ef2-2dd8-4f53-b2f9-9776a2150808 browser.cache.use_new_backend is set to 0

Wonder if this got worse now that we we switched away from the old backend. I've had at least two crashes recently bp-666b58a3-339a-454a-9e40-c12b52161020: @ nsImapCacheStreamListener::OnStopRequest https://dxr.mozilla.org/comm-central/rev/cf775e0371ea81bbd017c36c54d95c611ec44e86/mailnews/imap/src/nsImapProtocol.cpp#8726 bp-a3ec1a6f-09f4-4339-8894-b01762161014: @ nsImapCacheStreamListener::OnStopRequest (but lines off) + related to that as it trails down to cache entries (OnCacheEntryAvailable) bp-c709ae41-0545-4282-bb03-c9fa02161020: @ nsOfflineStoreCompactState::OnStopRequest https://dxr.mozilla.org/comm-central/rev/cf775e0371ea81bbd017c36c54d95c611ec44e86/mailnews/base/src/nsMsgFolderCompactor.cpp#1106

magnus nightly build bp-02192508-a95e-41af-bdaa-c38f22170205 lussnig bp-e8587a1b-3eed-4f23-81ab-d4b5f2170213

(In reply to Magnus Melin from comment #16) > Wonder if this got worse now that we we switched away from the old backend. Magnus, what "old backend" are you referring to?

I assume he's referring to browser.cache.use_new_backend = 0

Yes, I was referring to using the new cache backend. The crash report is a bit hard to interpret, but apparently something in nsImapMockChannel::Close https://dxr.mozilla.org/comm-central/rev/424cdd3c5239dd57a3110b6468bb79ca9d005eaa/mailnews/imap/src/nsImapProtocol.cpp#8807

Mason, I've looked up your current crashes and they are completely different, and infrequent. (last one was 3 months ago bp- 2016-12-28 20:27:28 ) Have you changed your setup?

Hello Wayne, My mail/news setup has not changed ;-) I think the latest crash signatures you see are for bug 1323688 (which seems to have been fixed in SM 2.49) I've just compacted my main IMAP account, and the operation completed smoothly. Regards.

(In reply to Magnus Melin from bug 1316416 comment #51) > The compact crash should be bug 1158578. It's very common for me, happens > mostly on my mozilla folder which is fairly big. I suspect size/timing makes > it more common, and also I suspect it's related to the new cache backend we > started using. Yes, this is what concerns me about going live with 52.0. I've had it on the back burner, because I don't know how bad this will get (In reply to Magnus Melin from comment #16) > Wonder if this got worse now that we we switched away from the old backend. Seems likely. But I can't say for sure. cache2 was enabled 2016-08-30 via bug 1021843 in TB51.0a1. Crash rate is higher in recent months for reporters with compact in their comments. The following shows nightly crash count 55.0a1 > 54.0a1 > 53.0a1. But we have no september data, so for what follows I cannot provide 51.0a1 vs 50.0a1 comparison. For non-release versions in the last 6 months visit the following link, then click version tab, then sort on versions column https://crash-stats.mozilla.com/search/?user_comments=~ompact&release_channel=%21release&product=Thunderbird&date=%3E%3D2016-09-30T19%3A03%3A50.000Z&date=%3C2017-03-30T19%3A03%3A50.000Z&_sort=-date&_facets=signature&_facets=version&_columns=date&_columns=signature&_columns=version&_columns=build_id&_columns=platform&_columns=user_comments#facet-version rank count signature 1 14 js::SavedStacks::insertFrames 2 8 js::jit::InlineFrameIterator::resetOn 3 7 nsCOMPtr_base::assign_assuming_AddRef | nsImapMockChannel::Close 4 5 nsOfflineStoreCompactState::OnStopRequest 5 4 nsScriptSecurityManager::CanCreateWrapper collectively for version 52.0b4 these signatures total 90+ crashes which, if all the crash reports are compact related, rank coampact as a #2 crash - https://crash-stats.mozilla.com/topcrashers/?product=Thunderbird&version=52.0b4&_facets_size=200 (To put in perspective using statistics from release versions, 45.6.0 and 45.7.1 each had 10 only crash comments per month. 45.8.0 has been out 3 weeks and not a single compact comment yet - not sure what to make of that)

OK, we meet again here ;-) Looking at two reports at random https://crash-stats.mozilla.com/report/index/bc525551-ed76-41a3-a8bb-5ffb52170330 https://crash-stats.mozilla.com/report/index/e40728e3-1e9d-49f4-bb8b-69a672170329 I see the same thing: Crashed at nsMsgMailSession.cpp:147 NS_IMETHODIMP nsMsgMailSession::OnItemPropertyFlagChanged(nsIMsgDBHdr *aItem, nsIAtom *aProperty, uint32_t aOldValue, uint32_t aNewValue) { NOTIFY_FOLDER_LISTENERS(propertyFlagChanged, OnItemPropertyFlagChanged, <=== 147 (aItem, aProperty, aOldValue, aNewValue)); return NS_OK; } The macro is this: #define NOTIFY_FOLDER_LISTENERS(propertyflag_, propertyfunc_, params_) \ PR_BEGIN_MACRO \ nsTObserverArray<folderListener>::ForwardIterator iter(mListeners); \ while (iter.HasMore()) { \ const folderListener &fL = iter.GetNext(); \ if (fL.mNotifyFlags & nsIFolderListener::propertyflag_) \ fL.mListener->propertyfunc_ params_; \ } \ PR_END_MACRO The crashes happen at the XPCOM boundary, since we seem to he calling a listener implemented in JS. I think "crash master" Kent needs to help us here ;-) I should mention for Kent that the crash signature has some JS details and is tracked in bug 1316416 which sadly is a mixed bag of all things that end up in JS processing due to various causes. The challenge is to eliminate these causes one by one, starting here ;-)

At least one error I looked at https://crash-stats.mozilla.com/report/index/e40728e3-1e9d-49f4-bb8b-69a672170329 is a stack overflow, which could make sense with the deep stack. There's a repetitive cycle of nsImapMockChannel::OpenCacheEntry() (called 5 times) that is suspicious. So I see two possibilities. One, you are in some sort of loop, and need to figure out why. Two, all of those calls are legitimate, but combined they overflow the stack. Then you need to exit at some point to stop the stack growth and add a call to post the request to the main window event loop rather than doing things like "return cacheStorage->AsyncOpenURI(newUri, extension, cacheAccess, this);"

(In reply to Kent James (:rkent) from comment #25) > There's a repetitive cycle of nsImapMockChannel::OpenCacheEntry() > (called 5 times) that is suspicious. Yes, I see that now in both of these: https://crash-stats.mozilla.com/report/index/bc525551-ed76-41a3-a8bb-5ffb52170330 https://crash-stats.mozilla.com/report/index/e40728e3-1e9d-49f4-bb8b-69a672170329 They all come from nsImapProtocol.cpp:9346: // This is where is gets complicated. The entire message is in the cache, // but we don't know whether it's suitable for use. Its meta data // might indicate that the message is incomplete. mTryingToReadPart = true; return cacheStorage->AsyncOpenURI(newUri, extension, cacheAccess, this); Surely that code has changed 500% in bug 1021843 and for the parts in bug 629738. What I don't understand is that three calls from there, we're at the same spot without ever going through OnCacheEntryAvailable(). That does appear later in the stack and it always calls ReadFromImapConnection() which means that the cache entry was indeed not in the cache or in the cache and unusable. I think what's going on here well be hard to understand without a reproducible case. So basically there seems to be some logic error in the interaction of cache2 with compaction. User lussnig talked about a "race condition" in bug 1316416 comment #47 and he may not be far off. I might play around with compacting IMAP folders, trying both with and without offline use. Maybe there is some message with an unexpected parts arrangement. But then Magnus crashes on his bugmail folder and these messages don't even have parts, or do you get your bugmail as HTML, Magnus?

Perhaps bienvenu's bug 619390 comment 3 is useful My guess is that this happens when we're compacting an offline store, and a message in the folder is not in the offline store, but is in the disk/memory cache, so we try to stream it to write to the new, compacted offline store. I've tried to reproduce that scenario but it hasn't led to any crashes. But that scenario is a bit tricky to reproduce, so I'll have to do a bit of debugging to see if I've actually reproduced it. Normally, we don't hit the ::Close() call in the destructor of the mock channel, and I haven't reproduced that.

Caught it in debugger. I'll have to look at the trace sometime later (the build is not trunk, a bit old, mostly still it should be the same). Thread 1 "thunderbird" received signal SIGSEGV, Segmentation fault. nsCOMPtr_base::~nsCOMPtr_base (this=this@entry=0x7fffffff8d90, __in_chrg=<optimized out>) at /opt/comm-central/src/obj-x86_64-pc-linux-gnu/dist/include/nsCOMPtr.h:294 294 NSCAP_RELEASE(this, mRawPtr); (gdb) bt #0 nsCOMPtr_base::~nsCOMPtr_base (this=this@entry=0x7fffffff8d90, __in_chrg=<optimized out>) at /opt/comm-central/src/obj-x86_64-pc-linux-gnu/dist/include/nsCOMPtr.h:294 #1 0x00007fffe941fb4a in nsOfflineStoreCompactState::CopyNextMessage ( this=this@entry=0x7fffbeaa3000, done=@0x7fffffff8e13: false) at /opt/comm-central/src/obj-x86_64-pc-linux-gnu/dist/include/nsCOMPtr.h:801 #2 0x00007fffe941fd60 in nsOfflineStoreCompactState::OnStopRequest ( this=0x7fffbeaa3000, request=<optimized out>, ctxt=<optimized out>, status=<optimized out>) at /opt/comm-central/src/mailnews/base/src/nsMsgFolderCompactor.cpp:1104 #3 0x00007fffe9530a1b in nsImapMockChannel::ReadFromImapConnection ( this=this@entry=0x7fff990e0b70) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9472 #4 0x00007fffe9532b47 in nsImapMockChannel::OnCacheEntryAvailable ( this=0x7fff990e0b70, entry=0x0, aNew=<optimized out>, aAppCache=<optimized out>, status=<optimized out>) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9170 #5 0x00007fffe98c4d0a in mozilla::net::CacheEntry::InvokeAvailableCallback ( this=this@entry=0x7fffa68a1ec0, aCallback=...) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:875 #6 0x00007fffe98c51bb in mozilla::net::CacheEntry::InvokeCallback ( this=this@entry=0x7fffa68a1ec0, aCallback=...) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:812 #7 0x00007fffe98c3be7 in mozilla::net::CacheEntry::InvokeCallbacks ( this=this@entry=0x7fffa68a1ec0, aReadOnly=aReadOnly@entry=true) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:678 #8 0x00007fffe98c3d01 in mozilla::net::CacheEntry::InvokeCallbacks ( this=0x7fffa68a1ec0) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:620 #9 0x00007fffe98c59b7 in mozilla::net::CacheEntry::Open ( this=this@entry=0x7fffa68a1ec0, aCallback=..., aTruncate=aTruncate@entry=false, aPriority=aPriority@entry=false, aBypassIfBusy=aBypassIfBusy@entry=false) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:338 #10 0x00007fffe98c5abf in mozilla::net::CacheEntry::AsyncOpen ( this=0x7fffa68a1ec0, aCallback=aCallback@entry=0x7fff990e0b78, aFlags=aFlags@entry=2) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:312 #11 0x00007fffe98d411c in mozilla::net::CacheStorage::AsyncOpenURI ( this=0x7fffc0fd15e0, aURI=<optimized out>, aIdExtension=..., aFlags=2, aCallback=0x7fff990e0b78) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheStorage.cpp:112 ---Type <return> to continue, or q <return> to quit--- #12 0x00007fffe9531316 in nsImapMockChannel::OpenCacheEntry ( this=this@entry=0x7fff990e0b70) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9287 #13 0x00007fffe953177a in nsImapMockChannel::AsyncOpen (this=0x7fff990e0b70, listener=<optimized out>, ctxt=0x7fffb49a88c8) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9610 #14 0x00007fffe954ad46 in nsImapService::GetMessageFromUrl ( this=this@entry=0x7fffd3d61510, aImapUrl=<optimized out>, aImapAction=aImapAction@entry=268435510, aImapMailFolder=<optimized out>, aImapMessage=<optimized out>, aMsgWindow=aMsgWindow@entry=0x7fffc7f78ba0, aDisplayConsumer=0x7fffbeaa3000, aConvertDataToText=false, aURL=0x7fffffff9b38) at /opt/comm-central/src/mailnews/imap/src/nsImapService.cpp:1109 #15 0x00007fffe954b4d9 in nsImapService::StreamMessage (this=0x7fffd3d61510, aMessageURI=<optimized out>, aConsumer=0x7fffbeaa3000, aMsgWindow=0x7fffc7f78ba0, aUrlListener=<optimized out>, aConvertData=<optimized out>, aAdditionalHeader=..., aLocalOnly=true, aURL=0x7fffffff9b38) at /opt/comm-central/src/mailnews/imap/src/nsImapService.cpp:1210 #16 0x00007fffe941fad3 in nsOfflineStoreCompactState::CopyNextMessage ( this=this@entry=0x7fffbeaa3000, done=@0x7fffffff9bb3: false) at /opt/comm-central/src/mailnews/base/src/nsMsgFolderCompactor.cpp:1037 #17 0x00007fffe941fd60 in nsOfflineStoreCompactState::OnStopRequest ( this=0x7fffbeaa3000, request=<optimized out>, ctxt=<optimized out>, status=<optimized out>) at /opt/comm-central/src/mailnews/base/src/nsMsgFolderCompactor.cpp:1104 #18 0x00007fffe9530a1b in nsImapMockChannel::ReadFromImapConnection ( this=this@entry=0x7fff990e0aa0) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9472 #19 0x00007fffe9532b47 in nsImapMockChannel::OnCacheEntryAvailable ( this=0x7fff990e0aa0, entry=0x0, aNew=<optimized out>, aAppCache=<optimized out>, status=<optimized out>) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9170 #20 0x00007fffe98c4d0a in mozilla::net::CacheEntry::InvokeAvailableCallback ( this=this@entry=0x7fffa1df76e0, aCallback=...) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:875 #21 0x00007fffe98c51bb in mozilla::net::CacheEntry::InvokeCallback ( this=this@entry=0x7fffa1df76e0, aCallback=...) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:812 #22 0x00007fffe98c3be7 in mozilla::net::CacheEntry::InvokeCallbacks ( this=this@entry=0x7fffa1df76e0, aReadOnly=aReadOnly@entry=true) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:678 #23 0x00007fffe98c3d01 in mozilla::net::CacheEntry::InvokeCallbacks ( this=0x7fffa1df76e0) ---Type <return> to continue, or q <return> to quit--- at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:620 #24 0x00007fffe98c59b7 in mozilla::net::CacheEntry::Open ( this=this@entry=0x7fffa1df76e0, aCallback=..., aTruncate=aTruncate@entry=false, aPriority=aPriority@entry=false, aBypassIfBusy=aBypassIfBusy@entry=false) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:338 #25 0x00007fffe98c5abf in mozilla::net::CacheEntry::AsyncOpen ( this=0x7fffa1df76e0, aCallback=aCallback@entry=0x7fff990e0aa8, aFlags=aFlags@entry=2) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:312 #26 0x00007fffe98d411c in mozilla::net::CacheStorage::AsyncOpenURI ( this=0x7fffc0fd15e0, aURI=<optimized out>, aIdExtension=..., aFlags=2, aCallback=0x7fff990e0aa8) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheStorage.cpp:112 #27 0x00007fffe9531316 in nsImapMockChannel::OpenCacheEntry ( this=this@entry=0x7fff990e0aa0) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9287 #28 0x00007fffe953177a in nsImapMockChannel::AsyncOpen (this=0x7fff990e0aa0, listener=<optimized out>, ctxt=0x7fffb49a82f8) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9610 #29 0x00007fffe954ad46 in nsImapService::GetMessageFromUrl ( this=this@entry=0x7fffd3d61510, aImapUrl=<optimized out>, aImapAction=aImapAction@entry=268435510, aImapMailFolder=<optimized out>, aImapMessage=<optimized out>, aMsgWindow=aMsgWindow@entry=0x7fffc7f78ba0, aDisplayConsumer=0x7fffbeaa3000, aConvertDataToText=false, aURL=0x7fffffffa8d8) at /opt/comm-central/src/mailnews/imap/src/nsImapService.cpp:1109 #30 0x00007fffe954b4d9 in nsImapService::StreamMessage (this=0x7fffd3d61510, aMessageURI=<optimized out>, aConsumer=0x7fffbeaa3000, aMsgWindow=0x7fffc7f78ba0, aUrlListener=<optimized out>, aConvertData=<optimized out>, aAdditionalHeader=..., aLocalOnly=true, aURL=0x7fffffffa8d8) at /opt/comm-central/src/mailnews/imap/src/nsImapService.cpp:1210 #31 0x00007fffe941fad3 in nsOfflineStoreCompactState::CopyNextMessage ( this=this@entry=0x7fffbeaa3000, done=@0x7fffffffa953: false) at /opt/comm-central/src/mailnews/base/src/nsMsgFolderCompactor.cpp:1037 #32 0x00007fffe941fd60 in nsOfflineStoreCompactState::OnStopRequest ( this=0x7fffbeaa3000, request=<optimized out>, ctxt=<optimized out>, status=<optimized out>) at /opt/comm-central/src/mailnews/base/src/nsMsgFolderCompactor.cpp:1104 #33 0x00007fffe9530a1b in nsImapMockChannel::ReadFromImapConnection ( this=this@entry=0x7fff990e09d0) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9472 #34 0x00007fffe9532b47 in nsImapMockChannel::OnCacheEntryAvailable ( ---Type <return> to continue, or q <return> to quit--- this=0x7fff990e09d0, entry=0x0, aNew=<optimized out>, aAppCache=<optimized out>, status=<optimized out>) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9170 #35 0x00007fffe98c4d0a in mozilla::net::CacheEntry::InvokeAvailableCallback ( this=this@entry=0x7fffa3874b40, aCallback=...) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:875 #36 0x00007fffe98c51bb in mozilla::net::CacheEntry::InvokeCallback ( this=this@entry=0x7fffa3874b40, aCallback=...) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:812 #37 0x00007fffe98c3be7 in mozilla::net::CacheEntry::InvokeCallbacks ( this=this@entry=0x7fffa3874b40, aReadOnly=aReadOnly@entry=true) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:678 #38 0x00007fffe98c3d01 in mozilla::net::CacheEntry::InvokeCallbacks ( this=0x7fffa3874b40) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:620 #39 0x00007fffe98c59b7 in mozilla::net::CacheEntry::Open ( this=this@entry=0x7fffa3874b40, aCallback=..., aTruncate=aTruncate@entry=false, aPriority=aPriority@entry=false, aBypassIfBusy=aBypassIfBusy@entry=false) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:338 #40 0x00007fffe98c5abf in mozilla::net::CacheEntry::AsyncOpen ( this=0x7fffa3874b40, aCallback=aCallback@entry=0x7fff990e09d8, aFlags=aFlags@entry=2) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:312 #41 0x00007fffe98d411c in mozilla::net::CacheStorage::AsyncOpenURI ( this=0x7fffc0fd15e0, aURI=<optimized out>, aIdExtension=..., aFlags=2, aCallback=0x7fff990e09d8) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheStorage.cpp:112 #42 0x00007fffe9531316 in nsImapMockChannel::OpenCacheEntry ( this=this@entry=0x7fff990e09d0) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9287 #43 0x00007fffe953177a in nsImapMockChannel::AsyncOpen (this=0x7fff990e09d0, listener=<optimized out>, ctxt=0x7fffb49a5478) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9610 #44 0x00007fffe954ad46 in nsImapService::GetMessageFromUrl ( this=this@entry=0x7fffd3d61510, aImapUrl=<optimized out>, aImapAction=aImapAction@entry=268435510, aImapMailFolder=<optimized out>, aImapMessage=<optimized out>, aMsgWindow=aMsgWindow@entry=0x7fffc7f78ba0, aDisplayConsumer=0x7fffbeaa3000, aConvertDataToText=false, aURL=0x7fffffffb678) at /opt/comm-central/src/mailnews/imap/src/nsImapService.cpp:1109 #45 0x00007fffe954b4d9 in nsImapService::StreamMessage (this=0x7fffd3d61510, aMessageURI=<optimized out>, aConsumer=0x7fffbeaa3000, aMsgWindow=0x7fffc7f78ba0, aUrlListener=<optimized out>, ---Type <return> to continue, or q <return> to quit--- at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:620 aConvertData=<optimized out>, aAdditionalHeader=..., aLocalOnly=true, aURL=0x7fffffffb678) at /opt/comm-central/src/mailnews/imap/src/nsImapService.cpp:1210 #46 0x00007fffe941fad3 in nsOfflineStoreCompactState::CopyNextMessage ( this=this@entry=0x7fffbeaa3000, done=@0x7fffffffb6f3: false) at /opt/comm-central/src/mailnews/base/src/nsMsgFolderCompactor.cpp:1037 #47 0x00007fffe941fd60 in nsOfflineStoreCompactState::OnStopRequest ( this=0x7fffbeaa3000, request=<optimized out>, ctxt=<optimized out>, status=<optimized out>) at /opt/comm-central/src/mailnews/base/src/nsMsgFolderCompactor.cpp:1104 #48 0x00007fffe9530a1b in nsImapMockChannel::ReadFromImapConnection ( this=this@entry=0x7fff990e0900) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9472 #49 0x00007fffe9532b47 in nsImapMockChannel::OnCacheEntryAvailable ( this=0x7fff990e0900, entry=0x0, aNew=<optimized out>, aAppCache=<optimized out>, status=<optimized out>) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9170 #50 0x00007fffe98c4d0a in mozilla::net::CacheEntry::InvokeAvailableCallback ( this=this@entry=0x7fffa1df9f20, aCallback=...) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:875 #51 0x00007fffe98c51bb in mozilla::net::CacheEntry::InvokeCallback ( this=this@entry=0x7fffa1df9f20, aCallback=...) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:812 #52 0x00007fffe98c3be7 in mozilla::net::CacheEntry::InvokeCallbacks ( this=this@entry=0x7fffa1df9f20, aReadOnly=aReadOnly@entry=true) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:678 #53 0x00007fffe98c3d01 in mozilla::net::CacheEntry::InvokeCallbacks ( this=0x7fffa1df9f20) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:620 #54 0x00007fffe98c59b7 in mozilla::net::CacheEntry::Open ( this=this@entry=0x7fffa1df9f20, aCallback=..., aTruncate=aTruncate@entry=false, aPriority=aPriority@entry=false, aBypassIfBusy=aBypassIfBusy@entry=false) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:338 #55 0x00007fffe98c5abf in mozilla::net::CacheEntry::AsyncOpen ( this=0x7fffa1df9f20, aCallback=aCallback@entry=0x7fff990e0908, aFlags=aFlags@entry=2) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheEntry.cpp:312 #56 0x00007fffe98d411c in mozilla::net::CacheStorage::AsyncOpenURI ( this=0x7fffc0fd15e0, aURI=<optimized out>, aIdExtension=..., aFlags=2, aCallback=0x7fff990e0908) at /opt/comm-central/src/mozilla/netwerk/cache2/CacheStorage.cpp:112 #57 0x00007fffe9531316 in nsImapMockChannel::OpenCacheEntry ( this=this@entry=0x7fff990e0900) ---Type <return> to continue, or q <return> to quit---#24 0x00007fffe98c59b7 in mozilla::net::CacheEntry::Open ( at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9287 #58 0x00007fffe953177a in nsImapMockChannel::AsyncOpen (this=0x7fff990e0900, listener=<optimized out>, ctxt=0x7fffa68f9ea8) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:9610 #59 0x00007fffe954ad46 in nsImapService::GetMessageFromUrl ( this=this@entry=0x7fffd3d61510, aImapUrl=<optimized out>, aImapAction=aImapAction@entry=268435510, aImapMailFolder=<optimized out>, aImapMessage=<optimized out>, aMsgWindow=aMsgWindow@entry=0x7fffc7f78ba0, aDisplayConsumer=0x7fffbeaa3000, aConvertDataToText=false, aURL=0x7fffffffc418) at /opt/comm-central/src/mailnews/imap/src/nsImapService.cpp:1109 #60 0x00007fffe954b4d9 in nsImapService::StreamMessage (this=0x7fffd3d61510, aMessageURI=<optimized out>, aConsumer=0x7fffbeaa3000, aMsgWindow=0x7fffc7f78ba0, aUrlListener=<optimized out>, aConvertData=<optimized out>, aAdditionalHeader=..., aLocalOnly=true, aURL=0x7fffffffc418) at /opt/comm-central/src/mailnews/imap/src/nsImapService.cpp:1210 #61 0x00007fffe941fad3 in nsOfflineStoreCompactState::CopyNextMessage ( this=this@entry=0x7fffbeaa3000, done=@0x7fffffffc493: false) at /opt/comm-central/src/mailnews/base/src/nsMsgFolderCompactor.cpp:1037 #62 0x00007fffe941fd60 in nsOfflineStoreCompactState::OnStopRequest ( this=0x7fffbeaa3000, request=<optimized out>, ctxt=<optimized out>, status=<optimized out>) at /opt/comm-central/src/mailnews/base/src/nsMsgFolderCompactor.cpp:1104 #63 0x00007fffe952c9a2 in nsImapCacheStreamListener::OnStopRequest ( this=0x7fffa77c6a00, request=<optimized out>, aCtxt=<optimized out>, aStatus=nsresult::NS_OK) at /opt/comm-central/src/mailnews/imap/src/nsImapProtocol.cpp:8810 #64 0x00007fffe96f769a in nsInputStreamPump::OnStateStop (this=0x7fffa8e140d0) at /opt/comm-central/src/mozilla/netwerk/base/nsInputStreamPump.cpp:715 #65 0x00007fffe96fca96 in nsInputStreamPump::OnInputStreamReady ( this=0x7fffa8e140d0, stream=<optimized out>) at /opt/comm-central/src/mozilla/netwerk/base/nsInputStreamPump.cpp:433 #66 0x00007fffe967bd2b in nsInputStreamReadyEvent::Run (this=0x7fffa3cdcdd0) at /opt/comm-central/src/mozilla/xpcom/io/nsStreamUtils.cpp:96 #67 0x00007fffe96936f3 in nsThread::ProcessNextEvent (this=0x7ffff6b51250, aMayWait=<optimized out>, aResult=0x7fffffffc797) at /opt/comm-central/src/mozilla/xpcom/threads/nsThread.cpp:1269 #68 0x00007fffe9693f7a in NS_ProcessNextEvent (aThread=<optimized out>, aThread@entry=0x7ffff6b51250, aMayWait=aMayWait@entry=false) at /opt/comm-central/src/mozilla/xpcom/threads/nsThreadUtils.cpp:389 #69 0x00007fffe9a266a1 in mozilla::ipc::MessagePump::Run (this=0x7fffe64f3040, aDelegate=0x7ffff6b6c260) at /opt/comm-central/src/mozilla/ipc/glue/MessagePump.cpp:96 ---Type <return> to continue, or q <return> to quit--- this=this@entry=0x7fffa1df76e0, aCallback=..., #70 0x00007fffe99e986c in MessageLoop::RunHandler (this=<optimized out>) at /opt/comm-central/src/mozilla/ipc/chromium/src/base/message_loop.cc:231 #71 MessageLoop::Run (this=<optimized out>) at /opt/comm-central/src/mozilla/ipc/chromium/src/base/message_loop.cc:211 #72 0x00007fffeacd2e11 in nsBaseAppShell::Run (this=0x7fffdde54560) at /opt/comm-central/src/mozilla/widget/nsBaseAppShell.cpp:156 #73 0x00007fffeb94f0ea in nsAppStartup::Run (this=0x7fffdde2e3d0) at /opt/comm-central/src/mozilla/toolkit/components/startup/nsAppStartup.cpp:283 #74 0x00007fffeb9b5906 in XREMain::XRE_mainRun (this=this@entry=0x7fffffffcac0) at /opt/comm-central/src/mozilla/toolkit/xre/nsAppRunner.cpp:4492 #75 0x00007fffeb9b5f10 in XREMain::XRE_main (this=this@entry=0x7fffffffcac0, argc=argc@entry=1, argv=argv@entry=0x7fffffffddf8, aConfig=...) at /opt/comm-central/src/mozilla/toolkit/xre/nsAppRunner.cpp:4670 #76 0x00007fffeb9b61a3 in XRE_main (argc=1, argv=0x7fffffffddf8, aConfig=...) at /opt/comm-central/src/mozilla/toolkit/xre/nsAppRunner.cpp:4761 #77 0x0000000000405d8d in do_main (argc=argc@entry=1, argv=argv@entry=0x7fffffffddf8, envp=envp@entry=0x7fffffffde08) at /opt/comm-central/src/mail/app/nsMailApp.cpp:237 #78 0x0000000000405717 in main (argc=1, argv=0x7fffffffddf8, envp=0x7fffffffde08) at /opt/comm-central/src/mail/app/nsMailApp.cpp:308

magnus, did you have good results?

Very annoying bug, but unfortunately no :( My hunch is it's not the all regular "stuck in a loop" crash. It's doing a lot of stuff for many messages in a row. It's possible there is some missing "end" event that isn't being called so it consumes resources until it hits the roof. Maybe.

IT is good that you are crashing :) Some of your other crashes nsOfflineStoreCompactState::CopyNextMessage bp-3b89c399-08c2-44ff-aff3-a641d2170412 0 xul.dll nsOfflineStoreCompactState::CopyNextMessage(bool&) C:/builds/moz2_slave/tb-rel-c-esr52-w32_bld-0000000/build/mailnews/base/src/nsMsgFolderCompactor.cpp:1052 1 xul.dll nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, nsresult) C:/builds/moz2_slave/tb-rel-c-esr52-w32_bld-0000000/build/mailnews/base/src/nsMsgFolderCompactor.cpp:1104 2 xul.dll nsImapCacheStreamListener::OnStopRequest(nsIRequest*, nsISupports*, nsresult) C:/builds/moz2_slave/tb-rel-c-esr52-w32_bld-0000000/build/mailnews/imap/src/nsImapProtocol.cpp:8741 3 xul.dll nsInputStreamPump::OnStateStop() netwerk/base/nsInputStreamPump.cpp:714 4 xul.dll nsInputStreamPump::OnInputStreamReady(nsIAsyncInputStream*) netwerk/base/nsInputStreamPump.cpp:433 nsOfflineStoreCompactState::OnStopRequest bp-9da6e670-2966-45fb-90d5-abfdc2170327 0 libxul.so nsOfflineStoreCompactState::OnStopRequest /builds/slave/tb-c-cen-l64-ntly-000000000000/build/mailnews/base/src/nsMsgFolderCompactor.cpp:1107 1 libxul.so nsImapCacheStreamListener::OnStopRequest /builds/slave/tb-c-cen-l64-ntly-000000000000/build/mailnews/imap/src/nsImapProtocol.cpp:8810 2 libxul.so nsInputStreamPump::OnStateStop netwerk/base/nsInputStreamPump.cpp:715 Whatever the cause, this is much worse in version 52 and severe enough to block unthrottling updates to all user. Please set the tracking flag

Let's track this bug here instead of bug 1316416. Repeating bug 1316416 comment #0: This is #1 crash for 51.0a2, perhaps beginning an upswing in early September. Fairly rare prior to that. Also rare on c-c, with no crashes since 2016-10-27. After the merge next week, need to evaluate a) whether the crashing on aurora stops, b) whether the crashing appears in beta. If it received an upswing in September 2016 that's because cache2 (bug 1021843) landed 2016-08-30. But the problem was certainly pre-existing as for example comment #4 or comment #12 from 2015 show. (In reply to Magnus Melin from comment #16) > Wonder if this got worse now that we we switched away from the old backend. Looks like it. Comment #25 to comment #27 are a good read.

I fixed bug 1355350 (pending review), so I have some cycles for this one. What are the STR? - IMAP folder (sync for offline or not? Apparently not, since only non-sync folders use cache). - Delete a whole lot of messages. - Compact, crash, right? Tried that, didn't crash. So let me know.

I don't know the exact cause. For me it's my mozilla folder, which is IMAP and synced for offline. I think the amount of messages plays in, mine is around 6000 messages. (Using mark-as-deleted if that possibly could make any difference.) Then delete some messages, compact the folder, wait for a few seconds. The actual expunge may or may not have the time to succeed on the server. After a few seconds it will just freeze, and then there's a crash in maybe 10-15 seconds.

(In reply to Jorg K (GMT+2) from comment #34) I have approximately the same configuration as in comment #35: - IMAP folder - selected for offline use - deleted messages are marked as deleted - about 1000 messages in the folder For me, the crash does not happen every time. It happens in about 10% of cases.

I guess the bug 1358140 is dupe of this one. Please have a look in comment 1 what I've been trying to find out, if that's any help.

Thanks for the hint. The thing I don't understand is this: All the reporters say their folder is "selected for offline use" or "synced for offline". How on earth is it possible that the code goes through caching, see nsImapMockChannel::OnCacheEntryAvailable() followed by the involvement of the nsStreamListenerTee, which is supposed to write to the channel and the cache at the same time: https://dxr.mozilla.org/comm-central/rev/5e4e889f13eb1fd5091f60921a1566c661f2c630/mailnews/imap/src/nsImapProtocol.cpp#9165 Further up in the stack: nsImapMockChannel::AsyncOpen() and nsImapMockChannel::OpenCacheEntry(). As far as I know, when the message is stored in a local folder for offline use, there is no caching, well, the local folder is the cache in this case. What am I missing?

This is repeating reliably for me. I have 2 separate Fedora 26 Linux machines (home, office), and both behave the same way. This appeared when Fedora 25 upgraded from Thunderbird 45.x to Thunderbird-52.0. Then I upgraded to Fedora 26, and the issue did not go away. Lastly I tried currently available Thunderbird binary from mozilla.org at one of these machines, and the same thing happened. About this IMAP INBOX: - Highest message id 85933 - about 20 000 messages in the box (says mutt) I had folder setting enabling offline use. When I disabled that, no crashing, INBOX compacting is quick.

Additional note: It was infinite recursion that killed my Thunderbird. .... #43322 0x00007fffe76e34fd in () at ./thunderbird/libxul.so #43323 0x00007fffe786c64b in () at ./thunderbird/libxul.so #43324 0x00007fffe786f90a in () at ./thunderbird/libxul.so #43325 0x00007fffe77fdb60 in () at ./thunderbird/libxul.so #43326 0x00007fffe7810b42 in () at ./thunderbird/libxul.so #43327 0x00007fffe782839a in () at ./thunderbird/libxul.so #43328 0x00007fffe7ac44ab in () at ./thunderbird/libxul.so #43329 0x00007fffe7ab0175 in () at ./thunderbird/libxul.so #43330 0x00007fffe8a4703b in () at ./thunderbird/libxul.so #43331 0x00007fffe9025e36 in () at ./thunderbird/libxul.so #43332 0x00007fffe9070ba8 in () at ./thunderbird/libxul.so #43333 0x00007fffe9070e56 in () at ./thunderbird/libxul.so #43334 0x00007fffe90710d7 in XRE_main () at ./thunderbird/libxul.so #43335 0x0000000000404d3c in _start () (gdb)

Bug reporter speaking. I am currently using Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49 and I have not had any mail-related crashes recently, other than bug 1323688. Did TB and SM code diverge?

I am getting Crashes while compacting mails very often (since months). Now I can't compact anymore, because every time TB crashes without (before) compacting. I have Mozilla/5.0 (Windows NT 10.0; WOW64; rv:54.0) Gecko/20100101 Thunderbird/54.0a2 Here are my (compacting) crash reports of today: https://crash-stats.mozilla.com/report/index/7171a24b-0e11-4221-915a-21fbc0170421 https://crash-stats.mozilla.com/report/index/7171a24b-0e11-4221-915a-21fbc0170421 Now I have repaired the folder, because I know that this helps for some days, until the crashes will start again.

And here is a crash report from my mac (same inbox as in Comment 43), related to this bug: https://crash-stats.mozilla.com/report/index/6500ed3f-86fb-4736-bf78-9412a0170414 . Mostly (as today) I get no crash information, but it is the same problem: https://crash-stats.mozilla.com/report/index/fec71d89-049c-47ed-8d60-7ff2e0170421

js::jit::InlineFrameIterator::::resetOn is #1 crash for nightly

Looking at one of the crashes, here is what happens: nsOfflineStoreCompactState::OnStopRequest() is done and decides to move onto the next message: https://dxr.mozilla.org/comm-central/rev/5e4e889f13eb1fd5091f60921a1566c661f2c630/mailnews/base/src/nsMsgFolderCompactor.cpp#1129 CopyNextMessage() https://dxr.mozilla.org/comm-central/rev/5e4e889f13eb1fd5091f60921a1566c661f2c630/mailnews/base/src/nsMsgFolderCompactor.cpp#1061 calls SteramMessage() https://dxr.mozilla.org/comm-central/rev/5e4e889f13eb1fd5091f60921a1566c661f2c630/mailnews/imap/src/nsImapService.cpp#1210 And if the message is not stored locally, it will be retrieved from the server. And then it's downhill from there. Needs some more digging, but something seems fishy. Why fetch a message from the server when the task is to compact (not download!) the folder? Could the people experiencing the crash try to switch to off-line before compacting. That should/may stop those downloads which are failing. It also looks like something is wrong with the listeners. I don't see why nsStreamListenerTee::OnStopRequest(), involved in filling the cache, should call nsOfflineStoreCompactState::OnStopRequest(), which causes the endless recursion.

> > Could the people experiencing the crash try to switch to off-line before > compacting. That should/may stop those downloads which are failing. > This is a crash on IMAP - inherently online - and technically server based. Going offline defeats the purpose of compacting.

(In reply to Jacques Amar from comment #48) > > > > Could the people experiencing the crash try to switch to off-line before > > compacting. That should/may stop those downloads which are failing. > > > > This is a crash on IMAP - inherently online - and technically server based. > Going offline defeats the purpose of compacting. I assume what was meant with "offline" was offline syncing. For me this solved the issue: Right click inbox, select "synchronization" tab and deselect "Select his folder for offline use" => after this compacting worked If I reeneable offline use and try compacting again, the problem comes back again.

(In reply to Peter Peltonen from comment #49) > (In reply to Jacques Amar from comment #48) > > > > > > Could the people experiencing the crash try to switch to off-line before > > > compacting. That should/may stop those downloads which are failing. > > > > > > > This is a crash on IMAP - inherently online - and technically server based. > > Going offline defeats the purpose of compacting. > > I assume what was meant with "offline" was offline syncing. For me this > solved the issue: > > Right click inbox, select "synchronization" tab and deselect "Select his > folder for offline use" > > => after this compacting worked > > If I reeneable offline use and try compacting again, the problem comes back > again. Yep (and duh!). Sync off works as advertised here too.

Following my analysis in comment #47, this patch makes the compact crash 100% in the endless recursion that we are seeing. The problem is, as described, that the message that is requested by the compaction is not in the local synced folder, a strange condition, that seems to occur at times, I don't know how. In that case, IMAP tries to fetch the message from the server and that goes into endless recursion. So now that we have a 100% reliable test case without having to shuffle data around, the fix shouldn't be far off. To answer the previous questions: I meant: Go offline, pull out the LAN cable, disconnect the WiFi. This way the compaction cannot get the message from the server and there won't be a crash. But I noticed that compaction is not available in offline mode.

I think the real problem that produces this bug is that messages are not downloaded correctly. I have selected that messages always should be downloaded for offline use, so there is no reason why some messages shouldn't be there for offline use. It also makes clear why it works after rebuilding the index, because all messages are downloaded again at once and this works. So maybe to fix this bug it is better to fix the download problem/errors.

Summarising my findings for today: The compactor calls nsImapService::StreamMessage() asking for a *local* message, either from the local sync'ed folder or the memory cache. The local folder says that it has the message, but when we come to retrieve it, it can't be retrieved and we go onto fetching it from the server and caching it locally in memory. Simulating retrieval failure from the local sync'ed folder gives me a plethora of crashes. Sadly the initial function StreamMessage() doesn't pass on the information that only a *local* message is required. By the time the local retrieval fails, this information isn't available any more and an attempt is made to fetch from the server. If I read comment #52 correctly, "rebuilding the index" (how? - Repair folder?) and downloading again will avoid the problem since all local messages are really retrievable locally. To be continued. I think, erroneously fetching from the server is not so bad as long as it doesn't crash ;-)

(In reply to Jorg K (GMT+2) from comment #53) > Summarising my findings for today: > The compactor calls nsImapService::StreamMessage() asking for a *local* > message, either from the local sync'ed folder or the memory cache. The local > folder says that it has the message, but when we come to retrieve it, it > can't be retrieved and we go onto fetching it from the server and caching it > locally in memory. > > Simulating retrieval failure from the local sync'ed folder gives me a > plethora of crashes. > > Sadly the initial function StreamMessage() doesn't pass on the information > that only a *local* message is required. By the time the local retrieval > fails, this information isn't available any more and an attempt is made to > fetch from the server. Well, not completely, from backtrace reported by users I see the nsImapMockChannel::ReadFromImapConnection does not try to download message, because imapUrl->GetLocalFetchOnly(&localOnly) return localOnly=true, therefore it returns NS_MSG_ERROR_MSG_NOT_OFFLINE [1]. The NS_MSG_ERROR_MSG_NOT_OFFLINE is propagated to the nsOfflineStoreCompactState::OnStopRequest [2] (see the interesting comment [3] and the OnStopRequest later calls 'rv = CopyNextMessage(done);' [4]. Why is CopyNextMessage called from there? This cause the recursion in case the message failed to download because the previous call of CopyNextMessage is still on the stack. The CopyNextMessage [5] should already process all messages in processed folder by using while loop, shouldn't it? [1] https://dxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp#9492 [2] https://dxr.mozilla.org/comm-central/source/mailnews/base/src/nsMsgFolderCompactor.cpp#1086 [3] https://dxr.mozilla.org/comm-central/source/mailnews/base/src/nsMsgFolderCompactor.cpp#1095 [4] https://dxr.mozilla.org/comm-central/source/mailnews/base/src/nsMsgFolderCompactor.cpp#1129 [5] https://dxr.mozilla.org/comm-central/source/mailnews/base/src/nsMsgFolderCompactor.cpp?q=CopyNextMessage&redirect_type=direct#1031

Jan, thanks for the comment. At least my poking around and semi-correct findings are causing a discussion now ;-) Glad I gave up at midnight to make space for someone with an analytical eye. Would you like to propose a patch? Looks like we're getting closer.

This is the patch which will produce endless recursion simulating the case when the ReadFromLocalCache() fails.

Umm, I attached pretty much the same earlier. Now, how do we fix it?

Looking into the code a bit more: CopyNextMessage() is called twice, once in nsOfflineStoreCompactState::StartCompacting() and the second time in nsOfflineStoreCompactState::OnStopRequest() to kick off the next message. The while loop inside CopyNextMessage() breaks after successfully kicking off the streaming of a message, https://dxr.mozilla.org/comm-central/rev/5e4e889f13eb1fd5091f60921a1566c661f2c630/mailnews/base/src/nsMsgFolderCompactor.cpp#1077 so moving on to the next message is done in the OnStopRequest(). The while loop *cannot* process more than one message, since the streaming is asynchronous. So it kicks off one message and exists the function, to be called again when that message is done. There is not even any recursion here, since OnStopRequest() only kicks off the next message before returning and completing the async thread. I didn't write that code, but I guess this answers your question. That said, I still don't see how we get into the recursive situation. When if NS_MSG_ERROR_MSG_NOT_OFFLINE is passed to OnStopRequest(), 'm_curIndex' is increased, so we move to the next message.

(In reply to Jorg K (GMT+2) from comment #58) > Looking into the code a bit more: > CopyNextMessage() is called twice, once in > nsOfflineStoreCompactState::StartCompacting() and the second time in > nsOfflineStoreCompactState::OnStopRequest() to kick off the next message. > > The while loop inside CopyNextMessage() breaks after successfully kicking > off the streaming of a message, > https://dxr.mozilla.org/comm-central/rev/ > 5e4e889f13eb1fd5091f60921a1566c661f2c630/mailnews/base/src/ > nsMsgFolderCompactor.cpp#1077 > so moving on to the next message is done in the OnStopRequest(). > > The while loop *cannot* process more than one message, since the streaming > is asynchronous. So it kicks off one message and exists the function, to be > called again when that message is done. There is not even any recursion > here, since OnStopRequest() only kicks off the next message before returning > and completing the async thread. > > I didn't write that code, but I guess this answers your question. > > That said, I still don't see how we get into the recursive situation. When > if NS_MSG_ERROR_MSG_NOT_OFFLINE is passed to OnStopRequest(), 'm_curIndex' > is increased, so we move to the next message. Yes we move to next message, but we still have previous call of CopyNextMessage on stack while calling another CopyNextMessage(). I think the problem lies there: https://dxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapProtocol.cpp?q=%2Bfunction%3AnsImapMockChannel%3A%3AReadFromImapConnection%28%29&redirect_type=single#9499 The OnStopRequest is called immediately there while the CopyNextMessage is still on stack. This normally wouldn't happened because if the cache hits, the result arrives asynchronously and processed by OnStopRequest while we're already out of CopyNextMessage.

I don't think I agree. nsImapMockChannel::OpenCacheEntry() calls AsyncOpenURI() and that asynchronously transfers control to OnCacheEntryAvailable() which calls ReadFromImapConnection() which then calls OnStopRequest(). At the very latest CopyNextMessage() should have returned after OpenCacheEntry() returned. What am I missing? I must be missing something since the crash dumps published here all seem to show endless recursions. I noticed that a few times the recursion actually stops in nsMsgMailSession::OnItemPropertyFlagChanged() just before the crash: https://crash-stats.mozilla.com/report/index/bc525551-ed76-41a3-a8bb-5ffb52170330 from comment #26.

(In reply to Jorg K (GMT+2) from comment #60) > I must be missing something since the crash dumps published here all seem to > show endless recursions. In my case, originally reported in bug #1352334, there was no recursion. Also, I would like to confirm that doing "Repair Folder" stops the crashes for me, at least for a while.

(In reply to Jorg K (GMT+2) from comment #60) > I don't think I agree. nsImapMockChannel::OpenCacheEntry() calls > AsyncOpenURI() and that asynchronously transfers control to > OnCacheEntryAvailable() which calls ReadFromImapConnection() which then > calls OnStopRequest(). At the very latest CopyNextMessage() should have > returned after OpenCacheEntry() returned. What am I missing? Um...you're right, it's more complicated :). Looks like CacheEntry::Open (called by CacheEntry::AsyncOpen) "fails" because Load has a cache miss - Load returns true only when in LOADING state [1] and instead of returning the Open calls InvokeCallbacks. And this later leads to another CopyNextMessage by this backtrace: #0 nsOfflineStoreCompactState::CopyNextMessage(bool&) (this=, done=false) #1 nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, nsresult) #2 mozilla::net::nsStreamListenerTee::OnStopRequest(nsIRequest*, nsISupports*, nsresult) #3 nsImapMockChannel::ReadFromImapConnection() (this=) #4 nsImapMockChannel::OnCacheEntryAvailable(nsICacheEntry*, bool, nsIApplicationCache*, nsresult) #5 mozilla::net::CacheEntry::InvokeAvailableCallback(mozilla::net::CacheEntry::Callback const&) #6 mozilla::net::CacheEntry::InvokeCallback(mozilla::net::CacheEntry::Callback&) (this=, aCallback=...) #7 mozilla::net::CacheEntry::InvokeCallbacks(bool) (this=, aReadOnly=false) #8 mozilla::net::CacheEntry::InvokeCallbacks() (this=) #9 mozilla::net::CacheEntry::Open(mozilla::net::CacheEntry::Callback&, bool, bool, bool) #10 mozilla::net::CacheEntry::AsyncOpen(nsICacheEntryOpenCallback*, unsigned int) So callbacks are no longer called async. Suprisingly it's probably bad idea to expect that async calls always ends as a new event. [1] http://searchfox.org/mozilla-central/source/netwerk/cache2/CacheEntry.cpp#448 I've been able to fix/workaround this by: 1. remove direct call of m_channelListener->OnStopRequestcreating from nsImapMockChannel::ReadFromImapConnection 2. add a runnable class which is dispatched to the current thread which do the m_channelListener->OnStopRequestcreating call. See Btw this is how the backtrace should look like for subsequent calls of CopyNextMessage when the message is available offline: #0 nsOfflineStoreCompactState::CopyNextMessage(bool&) (this=0x7fff96d34000, done=@0x7fffffffbe3f: false) #1 nsOfflineStoreCompactState::OnStopRequest(nsIRequest*, nsISupports*, nsresult) #2 nsImapCacheStreamListener::OnStopRequest(nsIRequest*, nsISupports*, nsresult) #3 nsInputStreamPump::OnStateStop() (this=0x7fff7ece0ec0) #4 nsInputStreamPump::OnInputStreamReady(nsIAsyncInputStream*) #5 nsInputStreamReadyEvent::Run() (this=0x7fff805fa6f0) #6 nsThread::ProcessNextEvent(bool, bool*) (this=0x7ffff6bbd420, aMayWait=false, aResult=0x7fffffffc1df) #7 NS_ProcessNextEvent(nsIThread*, bool) (aThread=0x7ffff6bbd420, aMayWait=false) (backtrace are shortened for less verbose/fitting the message, I can attach full bt if you want). > I must be missing something since the crash dumps published here all seem to > show endless recursions. I noticed that a few times the recursion actually > stops in nsMsgMailSession::OnItemPropertyFlagChanged() just before the crash: > https://crash-stats.mozilla.com/report/index/bc525551-ed76-41a3-a8bb- > 5ffb52170330 from comment #26. Hm, I don't think it actually stopped, only the last processed message was available as offline and during that execution the stack overflow happened (see the frame number at the end of report).

That's the patch relevant to previous comment.

Comment on attachment 8861834 [details] [diff] [review] rather workaroung patch to call nsOfflineStoreCompactState::OnStopRequest from runnable Review of attachment 8861834 [details] [diff] [review]: ----------------------------------------------------------------- Nice job! Let me do a try build on Windows for people to test. ::: mailnews/imap/src/nsImapProtocol.cpp @@ +9505,4 @@ > return rv; > } > > +class nsReadFromImapConnectionFailure : public mozilla::Runnable How about some more comments here. @@ +9522,5 @@ > + RefPtr<nsImapMockChannel> mImapMockChannel; > +}; > + > + > +nsresult nsImapMockChannel::RunListeners() I suggest to change the name here, maybe: RunOnStopRequestFailure() ? @@ +9679,4 @@ > || imapAction == nsIImapUrl::nsImapMsgFetchPeek)) > return NS_ERROR_FAILURE; // abort the running of this url....it failed a security check > } > + /*TEMPORARILY DONT DO THAT TO REPRODUCE PROBLEM I'll compile a try build with this removed.

https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=c6a23ce8cbe1d22633af16b31d045fa7957e448c I'll publish the download link later.

Anyone keen to try can download a Windows build here: https://archive.mozilla.org/pub/thunderbird/try-builds/mozilla@jorgk.com-c6a23ce8cbe1d22633af16b31d045fa7957e448c/try-comm-central-win32/ Other platforms upon request.

Hi, i can check it in the evening :-)

Got with the new Build an crash and submitted it. Maybe this will help.

Kindly let us know the crash ID.

(In reply to lussnig from comment #69) > Got with the new Build an crash and submitted it. Maybe this will help. xul.dll@0x14b2b9 | xul.dll@0x14b364 | xul.dll@0x14c133 | xul.dll@0x23b3c8 | xul.dll@0x23cfaa | xul.dll@0x26ced1 | xul.dll@0x2a1bb4 | xul.dll@0x3aecb0 | xul.dll@0x3ae544 | xul.dll@0x1afec24 | xul.dll@0x1afeba8 | xul.dll@0x1c87baa | xul.dll@0x1c8658a | x... bp-958380fd-5851-4e9d-b2fa-8fd210170426

bp-958380fd-5851-4e9d-b2fa-8fd210170426.txt was mine

This crash report is sadly useless. Personally, I'd take this patch since it doesn't make matters worse. I haven't tested it, but I trust Jan has run it with the hunk that stops the retrieval from the local folder and would force the endless recursion unless mitigated by the patch. Jan, are you OK with my changes? This bug has now become a collection pool of all problems related to IMAP compaction, but we started off fixing the recursion problem. Or perhaps Magnus wants to do some testing here?

Fixed a comment.

(In reply to Jorg K (GMT+2) from comment #73) > This crash report is sadly useless. > > Personally, I'd take this patch since it doesn't make matters worse. I > haven't tested it, but I trust Jan has run it with the hunk that stops the > retrieval from the local folder and would force the endless recursion unless > mitigated by the patch. > > Jan, are you OK with my changes? > > This bug has now become a collection pool of all problems related to IMAP > compaction, but we started off fixing the recursion problem. > > Or perhaps Magnus wants to do some testing here? Looks good to me. Thanks for looking into it.

Comment on attachment 8862064 [details] [diff] [review] recursive-fix.patch - Jan's patch with a few tweaks (v2). Review of attachment 8862064 [details] [diff] [review]: ----------------------------------------------------------------- Compacted and it didn't crash, which it often but not always did before ::: mailnews/base/src/nsMsgFolderCompactor.cpp @@ +1094,5 @@ > > // The NS_MSG_ERROR_MSG_NOT_OFFLINE error should allow us to continue, so we > // check for it specifically and don't terminate the compaction. > + if (rv == NS_MSG_ERROR_MSG_NOT_OFFLINE) { > + NS_WARNING("Message should be available as offline but it is not"); can we add something to identify which message?

OK, as Magnus requested, now printing out the message that's not available. Actually, Magnus, you said it didn't crash, but did you see the warning?

https://hg.mozilla.org/comm-central/rev/2f6c5a34d8f821c85ddd8f4ccdec89004fd540b8 So let's see where the crash statistics go with this landed. I don't expect a 100% cure, but it won't make things worse either.

Comment on attachment 8862136 [details] [diff] [review] recursive-fix.patch - Jan's patch with a few tweaks (v3). Let's take this to TB 54 beta to see what it does ;-)

TB 54 (beta): https://hg.mozilla.org/releases/comm-beta/rev/e046c1af98b5cf74df0eb9d3a86c16cf3d26fdab

(In reply to Jorg K (GMT+2) from comment #77) > Actually, Magnus, you said it didn't crash, but did you see the warning? It wasn't a debug build so no warning would have been seen.

Sorry to interject ... Since the issue seems to be related to a corrupt local folder (fixing that folder fixed my crashes), would ther be a way to actually detect a corrupt folder and give the end user an indication that they should fix?

Jorg asked about the impact of the patch landed 2017-04-26. I can say with certainty * js::SavedStacks::insertFrames on nightly builds is gone starting with 04-27 build. The average had been 15 crashes per day. So this patch appears to kill Bug 1316416 * js::jit::InlineFrameIterator::resetOn on nightly builds is gone starting with 04-27 build. So this kills #1 nightly-only crash signature For other signatures we won't know whether there is impact until the patch ships in other versions, 54.0b1 and hopefully 52.1.1

Well, I asked whether the IMAP compact crash is gone now, which has been a long-standing issue, but aggravated by the switch to cache2 in bug 1021843. I think the answer is yes, so let's declare this bug here fixed. A very big thank you to Jan to analyse the problem and come up with a solution!! That said: Bug 1316416 is a collection of stack overruns caused by various issues, most prominently this issue here bug also readily reproducible by bug 1343536 (obscure scenario). So I leave it to Wayne to close bug 1343536.

Yes, many thanks to Jan for figuring this out!

Did this patch also land on osx? My windows thunderbird is no fine, but osx ist still crashing starting compacting: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:54.0) Gecko/20100101 Thunderbird/54.0a2 Crash reports are always empty, so I can't help with this (https://crash-stats.mozilla.com/report/index/2e07c1de-a90d-4706-aab1-150200170430).

(In reply to Marcel Kneuer from comment #86) > Did this patch also land on osx? My windows thunderbird is no fine, but osx > ist still crashing starting compacting: Mozilla/5.0 (Macintosh; Intel Mac OS > X 10.12; rv:54.0) Gecko/20100101 Thunderbird/54.0a2 > > Crash reports are always empty, so I can't help with this > (https://crash-stats.mozilla.com/report/index/2e07c1de-a90d-4706-aab1- > 150200170430). You are running aurora channel, but that channel is obsolete and didn't get the patch. Nightly channel has the patch https://archive.mozilla.org/pub/thunderbird/nightly/latest-comm-central/ Or change to beta which will get it soon.

TB 52 ESR: https://hg.mozilla.org/releases/comm-esr52/rev/a003e013e18bc2f4ba96717387ea86d23045ce2e

For those affected, 52.1.1 *candidate for release* is available for testing at Windows: https://archive.mozilla.org/pub/thunderbird/candidates/52.1.1-candidates/build1/win32/en-US/Thunderbird%20Setup%2052.1.1.exe Mac: https://archive.mozilla.org/pub/thunderbird/candidates/52.1.1-candidates/build1/mac/en-US/Thunderbird%2052.1.1.dmg Linux: https://archive.mozilla.org/pub/thunderbird/candidates/52.1.1-candidates/build1/linux-x86_64/en-US/thunderbird-52.1.1.tar.bz2

I tried 52.1.1 on my Mac, here my experiences: Good news: - I switched my inboxes to sync for offline use again - I deleted bunch of messages - I compacted the inboxes and now my TB didn't crash immediately like before, yay Bad news: - After a few hours of running I noticed Mozilla crash reporter and my TB gone, so it had crashed after some time - So I do not know if this is related to compacting or something else - Details of the submitted crash report: AdapterDeviceID: 0x08a0 AdapterVendorID: 0x10de Add-ons: sogo-connector%40inverse.ca:31.0.1,filtaquilla%40mesquilla.com:1.3.2,%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D:5.4.1.1,%7B972ce4c6-7e08-4474-a285-3208198ce6fd%7D:52.1.1 BuildID: 20170509142926 ContentSandboxCapable: 1 ContentSandboxLevel: 0 CrashTime: 1494854528 EMCheckCompatibility: true FramePoisonBase: 7ffffffff0dea000 FramePoisonSize: 4096 InstallTime: 1494851460 Notes: AdapterVendorID: 0x10de, AdapterDeviceID: 0x08a0FP(D00-L1100-W00000000-T0100) GL Context? GL Context+ ProductID: {3550f703-e582-4d05-9a08-453d09bdfdc6} ProductName: Thunderbird ReleaseChannel: release SafeMode: 0 SecondsSinceLastCrash: 16846 StartupCrash: 0 StartupTime: 1494851460 TelemetryEnvironment: {"build":{"applicationId":"{3550f703-e582-4d05-9a08-453d09bdfdc6}","applicationName":"Thunderbird","architecture":"x86-64","buildId":"20170509142926","version":"52.1.1","vendor":null,"platformVersion":"52.1.1","xpcomAbi":"x86_64-gcc3","hotfixVersion":null,"architecturesInBinary":"i386-x86_64"},"partner":{"distributionId":null,"distributionVersion":null,"partnerId":null,"distributor":null,"distributorChannel":null,"partnerNames":[]},"system":{"memoryMB":8192,"virtualMaxMB":null,"cpu":{"count":2,"cores":2,"vendor":"GenuineIntel","family":6,"model":23,"stepping":10,"l2cacheKB":3072,"l3cacheKB":null,"speedMHz":2400,"extensions":["hasMMX","hasSSE","hasSSE2","hasSSE3","hasSSSE3","hasSSE4_1"]},"os":{"name":"Darwin","version":"16.5.0","locale":"en-FI"},"hdd":{"profile":{"model":null,"revision":null},"binary":{"model":null,"revision":null},"system":{"model":null,"revision":null}},"gfx":{"D2DEnabled":null,"DWriteEnabled":null,"ContentBackend":"Skia","adapters":[{"description":null,"vendorID":"0x10de","deviceID":"0x08a0","subsysID":null,"RAM":null,"driver":null,"driverVersion":null,"driverDate":null,"GPUActive":true}],"monitors":[{"screenWidth":1280,"screenHeight":800,"scale":1}],"features":{"compositor":"none"}}},"settings":{"blocklistEnabled":true,"e10sEnabled":false,"e10sCohort":"unknown","telemetryEnabled":false,"locale":"en-US","update":{"channel":"release","enabled":true,"autoDownload":false},"userPrefs":{"app.update.auto":false,"browser.cache.disk.capacity":358400},"addonCompatibilityCheckEnabled":true,"isDefaultBrowser":null},"profile":{"creationDate":13725},"addons":{"activeAddons":{"sogo-connector@inverse.ca":{"blocklisted":false,"description":"A DAV plugin for keeping addressbooks and events in sync","name":"Inverse SOGo Connector","userDisabled":false,"appDisabled":false,"version":"31.0.1","scope":1,"type":"extension","foreignInstall":false,"hasBinaryComponents":false,"installDay":16482,"updateDay":16482,"isSystem":false},"filtaquilla@mesquilla.com":{"blocklisted":false,"description":"Mail filter custom actions and searches","name":"FiltaQuilla","userDisabled":false,"appDisabled":false,"version":"1.3.2","scope":1,"type":"extension","foreignInstall":false,"hasBinaryComponents":false,"installDay":17105,"updateDay":17105,"isSystem":false},"{e2fda1a4-762b-4020-b5ad-a41df1933103}":{"blocklisted":false,"description":"Integrated Calendaring & Scheduling for your Email client","name":"Lightning","userDisabled":false,"appDisabled":false,"version":"5.4.1.1","scope":1,"type":"extension","foreignInstall":false,"hasBinaryComponents":false,"installDay":14813,"updateDay":17301,"isSystem":false}},"theme":{"id":"{972ce4c6-7e08-4474-a285-3208198ce6fd}","blocklisted":false,"description":"The default theme.","name":"Default","userDisabled":false,"appDisabled":false,"version":"52.1.1","scope":4,"foreignInstall":false,"hasBinaryComponents":false,"installDay":15162,"updateDay":17296},"activePlugins":[],"activeGMPlugins":{},"activeExperiment":{},"persona":null}} Theme: classic/1.0 Throttleable: 1 UptimeTS: 3083.34533823 Vendor: Version: 52.1.1 useragent_locale: en-US This report also contains technical information about the state of the application when it crashed.

(In reply to Peter Peltonen from comment #91) > I tried 52.1.1 on my Mac, here my experiences: > > Good news: > - I switched my inboxes to sync for offline use again > - I deleted bunch of messages > - I compacted the inboxes and now my TB didn't crash immediately like > before, yay Thanks for that info. > Bad news: > - After a few hours of running I noticed Mozilla crash reporter and my TB > gone, so it had crashed after some time > - So I do not know if this is related to compacting or something else > - Details of the submitted crash report: Unfortunately, the "details" data is never of any use. We would need your crash ID https://support.mozilla.org/en-US/kb/mozilla-crash-reporter#w_viewing-crash-reports

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #92) > > Unfortunately, the "details" data is never of any use. We would need your > crash ID > https://support.mozilla.org/en-US/kb/mozilla-crash-reporter#w_viewing-crash- > reports I do not see any Mac crash reports for 52.1.1, so you may need to submit it at Help > Troubleshooting. Another possibility is the crash reporter failed.

Ok, did not know about the crash report id, thanks for clarification. Here is the crash report I think (did not resubmit anything, so strange you did not see it before): https://crash-stats.mozilla.com/report/index/b1652bd6-6720-4479-a6b5-2a9a30170515 I do have experienced similar crashes not related to compacting with 52.1.0 as well, I think this earlier from the same day was one: https://crash-stats.mozilla.com/report/index/605b97dc-0079-44b4-ba0a-b0cc80170515 So perhaps this is a separate bug? I don't understand those reports, so hard for me to say... Haven't happened after that first crash with 52.1.0. I will let you know if it does.

Different issue, Wayne, can you please file a bug for those crashes.

(In reply to Peter Peltonen from comment #94) > Ok, did not know about the crash report id, thanks for clarification. Here > is the crash report I think (did not resubmit anything, so strange you did > not see it before): > > https://crash-stats.mozilla.com/report/index/b1652bd6-6720-4479-a6b5- > 2a9a30170515 bug 1365136. And afaict unrelated to this bug

(I find no record of there ever being a js::jit::InlineFrameIterator::::resetOn, so removing) All of these signatures are gone in 52.1.1 [@ nsImapMockChannel::Close ] [@ nsOfflineStoreCompactState::CopyNextMessage ] [@ nsOfflineStoreCompactState::OnStopRequest ] [@ nsScriptSecurityManager::CanCreateWrapper ] [@ nsImapCacheStreamListener::OnStopRequest ] plus mozilla::mailnews::MsgDBReporter::GetPath bug 1353704 js::SavedStacks::insertFrames Bug 1316416 and some users had reported Empty signature. All totalled, I expect 11-15% of crashes wiped out by this patch. v.fixed Thanks Jan and Jorg and everyone involved!