Closed Bug 1294074 Opened 8 years ago Closed 1 year ago

Crash in MOZ_Z_inflate_table. Local cache disabled, imap COMPRESS=DEFLATE (Cyrus IMAP server)

Categories

(MailNews Core :: Networking: IMAP, defect)

x86_64
All
defect

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: richard.leger, Unassigned)

References

Details

(Keywords: crash, steps-wanted, Whiteboard: [closeme 2022-10-15][rare])

Crash Data

This bug was filed from the Socorro interface and is report bp-9ef57086-8826-4a22-8574-00ccb2160803. ============================================================= End-user was browsing in IMAP Inbox when Thunderbird crashed...
Hardware: x86 → x86_64
bp-9ef57086-8826-4a22-8574-00ccb2160803 0 xul.dll MOZ_Z_inflate_table modules/zlib/src/inftrees.c:238 1 xul.dll MOZ_Z_inflate modules/zlib/src/inflate.c:927 2 xul.dll NS_ConsumeStream(nsIInputStream*, unsigned int, nsACString_internal&) xpcom/io/nsStreamUtils.cpp:695 3 xul.dll nsMsgCompressIStream::DoInflation() c:/builds/moz2_slave/tb-rel-c-esr45-w32_bld-0000000/build/mailnews/base/util/nsMsgCompressIStream.cpp:66 4 xul.dll nsMsgLineStreamBuffer::ReadNextLine(nsIInputStream*, unsigned int&, bool&, nsresult*, bool) c:/builds/moz2_slave/tb-rel-c-esr45-w32_bld-0000000/build/mailnews/base/util/nsMsgLineBuffer.cpp:375 5 xul.dll nsImapProtocol::CreateNewLineFromSocket() c:/builds/moz2_slave/tb-rel-c-esr45-w32_bld-0000000/build/mailnews/imap/src/nsImapProtocol.cpp:4704 6 xul.dll nsImapServerResponseParser::GetNextLineForParser(char**) c:/builds/moz2_slave/tb-rel-c-esr45-w32_bld-0000000/build/mailnews/imap/src/nsImapServerResponseParser.cpp:88 7 xul.dll nsIMAPGenericParser::AdvanceToNextLine() c:/builds/moz2_slave/tb-rel-c-esr45-w32_bld-0000000/build/mailnews/imap/src/nsIMAPGenericParser.cpp:148 8 xul.dll nsImapServerResponseParser::msg_fetch_literal(bool, int) c:/builds/moz2_slave/tb-rel-c-esr45-w32_bld-0000000/build/mailnews/imap/src/nsImapServerResponseParser.cpp:3115 9 xul.dll nsImapServerResponseParser::msg_fetch_content(bool, int, char const*) c:/builds/moz2_slave/tb-rel-c-esr45-w32_bld-0000000/build/mailnews/imap/src/nsImapServerResponseParser.cpp:2158 10 xul.dll nsImapServerResponseParser::msg_fetch() c:/builds/moz2_slave/tb-rel-c-esr45-w32_bld-0000000/build/mailnews/imap/src/nsImapServerResponseParser.cpp:1349 11 mozglue.dll arena_malloc_small memory/mozjemalloc/jemalloc.c:4241 12 mozglue.dll arena_malloc memory/mozjemalloc/jemalloc.c:4301 13 mozglue.dll imalloc memory/mozjemalloc/jemalloc.c:4313 14 xul.dll NS_strtok(char const*, char**) xpcom/glue/nsCRTGlue.cpp:55 15 xul.dll nsImapServerResponseParser::ParseIMAPServerResponse(char const*, bool, char*) c:/builds/moz2_slave/tb-rel-c-esr45-w32_bld-0000000/build/mailnews/imap/src/nsImapServerResponseParser.cpp:204
Component: General → Networking: IMAP
Product: Thunderbird → MailNews Core
Version: 45 Branch → 45
Status: UNCONFIRMED → NEW
Ever confirmed: true
Keywords: crash
Whiteboard: [rare][need STR]
FYI, TB still crashing unexpectedly and randomely sometime to time when end-users are opening email messages... here are more crash reports for reference... Thunderbird 45.5.0 Crash Report [@ MOZ_Z_inflate_table ] bp-18244e89-aed6-48bc-aae6-7b57e2161129 29/11/2016 10:29 NR https://crash-stats.mozilla.com/report/index/18244e89-aed6-48bc-aae6-7b57e2161129 Note: End-user was opening an email Thunderbird 45.4.0 Crash Report [@ MOZ_Z_inflate_table ] bp-c0249e2e-81f1-44b0-b7e0-957612161109 09/11/2016 11:06 SD bp-70c46455-9aa0-4a7f-a10f-0df082161109 09/11/2016 11:49 IP - 1294074 - https://bugzilla.mozilla.org/show_bug.cgi?id=1294074 bp-ddcf76cd-9c5f-433f-89b3-0c0012161107 07/11/2016 12:42 SD Thunderbird 45.3.0 Crash Report [@ MOZ_Z_inflate_table ] bp-a5c68f37-996d-44fc-ab2d-791712161003 03/10/2016 14:16 SD bp-97e2d760-9ad0-4219-b969-75bb82160928 28/09/2016 09:51 MM bp-6f3cf204-26ba-49d6-a068-d50b32160922 22/09/2016 15:02 NR Thunderbird 45.2.0 Crash Report [@ MOZ_Z_inflate_table ] bp-786f03f8-0d83-4fd0-ad78-da1472160905 05/09/2016 09:53 MM bp-55961a8a-22ce-46aa-9f5a-e39062160830 30/08/2016 13:27 NR bp-af975047-3a2f-436e-9e4d-9b6f32160823 23/08/2016 17:55 MM bp-d4089639-8dea-4ec9-bab0-088742160815 15/08/2016 09:47 SD bp-49cbe824-1ce3-42b1-9689-399102160811 11/08/2016 16:56 MM bp-53334cb2-1a1c-4e2b-b1dd-da7222160803 03/08/2016 14:44 SD
FYI, still observed recently... Thunderbird 45.7.1 Crash Report [@ MOZ_Z_inflate_table ] bp-e63c800a-337c-44fa-ac5e-15b7f2170221
Depends on: 1264302
Still rare. Probably related to bug 1333038 nsMsgLineStreamBuffer::ReadNextLine
Depends on: 1333038
With version 60, the crash rate has roughly tripled
Depends on: 1516501
Depends on: 1581017

(In reply to Wayne Mery (:wsmwk) from comment #5)

With version 60, the crash rate has roughly tripled

It looks like version 68 crash rate is currently 4-5x higher than 60 (adjusted for percentage of users on both releases). Unfortunately I didn't document last year's crash rate numbers.

That said, it's not a topcrash - ranked #105 for version 68.3.1

(emailed one crash reporter, grigorov)

FYI encountered this crash in TB 82.0b2...
bp-a2e84fe4-26f2-44e0-9ad4-0a9140201028

Crash rate is now half what it was 6 months ago for both this crash and bug 1581017.

The decrease in crash rate begins in early November, which coincides with two events

  1. turning up the knob to move version 68 users to version 78
  2. shipping the 78.4.x series - 78.4.0 https://mzl.la/3dFNWPL, 78.4.1 https://mzl.la/3r7o4Tk , 78.4.3 https://mzl.la/3dFNWPL (78.4.2 did not ship IIRC)

Several crash patches land in 78.4.0 but ISTM none would impact this bug. But a possible explanation is the v78 crash rate might already be lower than 68 from an earlier patch that did not get uplifted to 68. That can't be assessed without comparing relative crash rates over time of each version, which crash-stats does not readily provide.

I also checked 6 months of beta crashes to attempt to find a timing - but no change. Ditto for bug 1581017.

(In reply to Richard Leger from comment #8)

FYI encountered this crash in TB 82.0b2...
bp-a2e84fe4-26f2-44e0-9ad4-0a9140201028

0 xul.dll MOZ_Z_inflate_table(<unnamed-tag>, unsigned short*, unsigned int, code**, unsigned int*, unsigned short*) modules/zlib/src/inftrees.c:236
1 xul.dll MOZ_Z_inflate(z_stream_s*, int) modules/zlib/src/inflate.c:949
2 xul.dll nsMsgCompressIStream::Read(char*, unsigned int, unsigned int*) comm/mailnews/base/src/nsMsgCompressIStream.cpp:159
3 xul.dll nsMsgLineStreamBuffer::ReadNextLine(nsIInputStream*, unsigned int&, bool&, nsresult*, bool) comm/mailnews/base/src/nsMsgLineBuffer.cpp:335
4 xul.dll nsImapProtocol::CreateNewLineFromSocket() comm/mailnews/imap/src/nsImapProtocol.cpp:4745
5 xul.dll nsImapServerResponseParser::GetNextLineForParser(char**) comm/mailnews/imap/src/nsImapServerResponseParser.cpp:83
6 xul.dll nsIMAPGenericParser::AdvanceToNextToken() comm/mailnews/imap/src/nsIMAPGenericParser.cpp:95
7 xul.dll nsImapServerResponseParser::ParseIMAPServerResponse(char const*, bool, char*) comm/mailnews/imap/src/nsImapServerResponseParser.cpp:184
8 xul.dll nsImapProtocol::FetchMessage(nsTString<char> const&, <unnamed-tag>, char const*, unsigned int, unsigned int, char*) comm/mailnews/imap/src/nsImapProtocol.cpp:3642
9 xul.dll nsImapProtocol::FetchTryChunking(nsTString<char> const&, <unnamed-tag>, bool, char*, unsigned int, bool) comm/mailnews/imap/src/nsImapProtocol.cpp:3692
10 xul.dll nsIMAPBodypart::GeneratePart(nsIMAPBodyShell*, bool, bool) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:412
11 xul.dll nsIMAPBodypartLeaf::Generate(nsIMAPBodyShell*, bool, bool) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:531
12 xul.dll nsIMAPBodypartMultipart::Generate(nsIMAPBodyShell*, bool, bool) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:893
13 xul.dll nsIMAPBodypartMultipart::Generate(nsIMAPBodyShell*, bool, bool) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:893
14 xul.dll nsIMAPBodypartMultipart::Generate(nsIMAPBodyShell*, bool, bool) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:893
15 xul.dll nsIMAPBodypartMessage::Generate(nsIMAPBodyShell*, bool, bool) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:764
16 xul.dll nsIMAPBodyShell::Generate(char*) comm/mailnews/imap/src/nsIMAPBodyShell.cpp:227
17 xul.dll nsImapProtocol::ProcessSelectedStateURL() comm/mailnews/imap/src/nsImapProtocol.cpp:2773
18 xul.dll nsImapProtocol::ProcessCurrentURL() comm/mailnews/imap/src/nsImapProtocol.cpp:1792
19 xul.dll nsImapProtocol::ImapThreadMainLoop() comm/mailnews/imap/src/nsImapProtocol.cpp:1433
20 xul.dll nsImapProtocol::Run() comm/mailnews/imap/src/nsImapProtocol.cpp:1116

I assume this happens when fetching a new message and not when reading an existing message in offline store? Also, it's from a server using compression, e.g., gmail? If reproducible on a given server or message, does setting mail.server.default.use_compress_deflate false fix the problem?
Looks like array bounds are being violated deep down in the mozilla zlib code when "inflating" a compressed imap response.

Richard, do you still encounter this crash signature?

And like your bug 1333031, the crash rate here is about 50% less compared to 6 months ago.

Flags: needinfo?(richard.leger)

(In reply to gene smith from comment #11)

I assume this happens when fetching a new message and not when reading an existing message in offline store?

It was not from offline store as my main imap account cache is disabled. If I submitted the report bp-a2e84fe4-26f2-44e0-9ad4-0a9140201028 I would have put a comment indicating the last steps prior crash if I notice it. Are you able to access such comment?

Also, it's from a server using compression, e.g., gmail? If reproducible on a given server or message, does setting mail.server.default.use_compress_deflate false fix the problem?

Not Gmail, but a Cyrus IMAP server (I think Gmail also use Cyrus), the crash was random and not reproducible at will.

Looks like array bounds are being violated deep down in the mozilla zlib code when "inflating" a compressed imap response.

Is decompression occurring on the fly or after message is downloaded? I am not sure how to check if the server use decompression would IMAP logs shows it? I do have some from previous bug issues.

Flags: needinfo?(richard.leger)

(In reply to Wayne Mery (:wsmwk) from comment #12)

Richard, do you still encounter this crash signature?

Since the I have upgraded TB couple of times will check if report came back or not... keep you posted...

Flags: needinfo?(richard.leger)

Is decompression occurring on the fly or after message is downloaded? I am not sure how to check if the server use decompression would IMAP logs shows it? I do have some from previous bug issues.

I guess you would say it happens on the fly since imap code in tb never see or has to deal with compressed data. It's all handled in the mozilla read and zlib library. All tb code does is enable it if it sees the capability COMPRESS=DEFLATE. Then it sends to the server a command called "COMPRESS DEFLATE" so you will see these in the log like this:

I/IMAP 0x7fb1768d6800:imap.gmail.com:A:SendData: 4 COMPRESS DEFLATE
D/IMAP ReadNextLine [rv=0x0 stream=0x7fb171c18ac0 nb=14 needmore=0]
I/IMAP 0x7fb1768d6800:imap.gmail.com:A:CreateNewLineFromSocket: 4 OK Success

Before this you will also see a capability response like this:

I/IMAP 0x7fb1768d6800:imap.gmail.com:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ...

(In reply to gene smith from comment #15)

I guess you would say it happens on the fly since imap code in tb never see or has to deal with compressed data. It's all handled in the mozilla read and zlib library. All tb code does is enable it if it sees the capability COMPRESS=DEFLATE. Then it sends to the server a command called "COMPRESS DEFLATE" so you will see these in the log like this:
I/IMAP 0x7fb1768d6800:imap.gmail.com:A:SendData: 4 COMPRESS DEFLATE
(...)
I/IMAP 0x7fb1768d6800:imap.gmail.com:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ...

Yes I see such lines in logs:

(...)
2021-01-09 16:07:11.876000 UTC - [Parent 9104: IMAP]: I/IMAP EATE-SPECIAL-USE URLAUTH URLAUTH=BINARY LOGINDISABLED COMPRESS=DEFLATE X-QUOTA=STORAGE X-QUOTA=MESSAGE X-QUOTA=X-ANNOTATION-STORAGE X-QUOTA=X-NUM-FOLDERS IDLE] Success (tls protection) SESSIONID=
<cyrus-(...)
(...)
2021-01-09 16:07:12.130000 UTC - [Parent 9104: IMAP]: I/IMAP 0000026C18757000:mail.****:A:SendData: 5 COMPRESS DEFLATE
(...)

Flags: needinfo?(richard.leger)
Keywords: steps-wanted
Whiteboard: [rare][need STR] → [rare]

Made contact with one crash reporter, Jassen. No useful info yet.

Severity: critical → S3
OS: Windows 7 → All
Summary: Crash in MOZ_Z_inflate_table → Crash in MOZ_Z_inflate_table. Local cache disabled, imap COMPRESS=DEFLATE (Cyrus IMAP server)

Just encountered in TB 91.11.0 on Windows 10 while browsing email in a sub-folder IMAP account... no cache... online mode...
bp-2916ff34-4676-426b-9ab9-be9220220711

Could be linked to high latency on the Internet connection that I currently encounter... email are slow to load in preview and when selecting some of them TB crashes...

Ping statistics for 62.3.75.133:
Packets: Sent = 18, Received = 18, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 85ms, Maximum = 336ms, Average = 189ms

No version 102 crashes, like

  • Bug 1684266 - Crash in [@ ImapMessageSinkProxy::GetMessageSizeFromDB
  • Bug 1649674 Crash in nsStreamConverter::OnDataAvailable (78+ edition) - imap thread and freed memory
Whiteboard: [rare] → [closeme 2022-10-15][rare]

Only one version 102 crash in past 6 months bp-86ac3c51-a240-446a-bbb4-527c90230118

There are rare version 102 crashes, but none newer, and no version 115 crashes

Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.