Closed
Bug 532415
Opened 15 years ago
Closed 15 years ago
280MB emails in Gmail produce 38GB All Mail file
Categories
(MailNews Core :: Networking: IMAP, defect)
Tracking
(Not tracked)
RESOLVED
DUPLICATE
of bug 537498
People
(Reporter: Happy.Cerberus, Unassigned)
References
(Blocks 1 open bug)
Details
Attachments
(2 files)
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; cs-CZ; rv:1.9.2b4) Gecko/20091126 SUSE/3.6b4-2.1 Firefox/3.6b4
Build Identifier: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.5) Gecko/20091122 SUSE/3.0.0-4.1 Thunderbird/3.0
I have 280MB of emails in Gmail, but the All Mail file has 38GB (after compacting).
Reproducible: Always
Steps to Reproduce:
1. add Gmail account using IMAP
2. watch the file grow
Comment 1•15 years ago
|
||
Do you mean bug 517466 ?
Reporter | ||
Comment 2•15 years ago
|
||
No, this happens on the first fetch.
Comment 3•15 years ago
|
||
Bug 499630 and bug 487992 are already fixed.
"Compact doesn't reduce file size" is possibly due to "deleted count=0".
Try next, please.
1. Show "Order Received" column(UID of mail) for [Gmal]/All Mail
2. Move mail of largest UID to [Gmail]/Trash
3. Move back the mail to [Gmail]/All Mail
4. "Compact" [Gmail]/All Mail via context menu
Will file size of "/.../[Gmail]/All Mail" be reduced?
Reporter | ||
Comment 4•15 years ago
|
||
No, it doesn't help, but the compacting ends very soon, so it could be that it just determines there's nothing to be done and continues.
I have a very stupid idea. Could it be that all emails are given the size of the largest one? In that case the 38GB would be relatively correct.
Comment 5•15 years ago
|
||
(In reply to comment #3)
> Bug 499630 and bug 487992 are already fixed.
noting for the record that these fixes should be in Simon's build per comment 0
and getting this out of the horrible General component
Component: General → Backend
Product: Thunderbird → MailNews Core
QA Contact: general → backend
Version: unspecified → 1.9.0 Branch
Moving to IMAP component though as it's a specific offline-storage issue, which also may be Gmail-specific if only the All Mail folder is affected.
> (In reply to comment #4) Could it be that all emails are given the size of
> the largest one? In that case the 38GB would be relatively correct.
The size of each e-mail should be tracked individually in the offline storage, thus I don't think that's the issue.
Blocks: tb-gmailWIP
Component: Backend → Networking: IMAP
QA Contact: backend → networking.imap
Version: 1.9.0 Branch → Trunk
Reporter | ||
Comment 7•15 years ago
|
||
> Moving to IMAP component though as it's a specific offline-storage issue, which
also may be Gmail-specific if only the All Mail folder is affected.
I have the same issue with the Inbox and Sent folder, but yes this is a Gmail-specific issue. I have 4 IMAP accounts, and Gmail is the only one with this issue.
Comment 8•15 years ago
|
||
(In reply to comment #7)
> I have 4 IMAP accounts, and Gmail is the only one with this issue.
What size is the 38GB? File size of ".../[Gmail]/All Mail"? Total file size of a local directory for a Gmail IMAP accout?
If latter, it's normal if you requested auto-sync for all Gmail's IMAP folders, because Gmail Label is presented as IMAP folder by Gmail IMAP.
Err, for 280MB used on Gmails server to reach 38GB synchronized, you'd need
each message to have 138 labels assigned... don't think that's realistic. :-\
Reporter | ||
Comment 10•15 years ago
|
||
I'm talking about file size of single files.
Interesting thing just happened. I managed to compact Sent and Inbox, and they now have reasonable sizes. All Mail still refuses to compact.
Reporter | ||
Comment 11•15 years ago
|
||
I think this is the same issue that I'm experiencing with another IMAP server (where the bug has different manifestation).
See: https://bugzilla.mozilla.org/show_bug.cgi?id=532360
I can confirm that Gmail folders have no newlines between emails.
Comment 12•15 years ago
|
||
(In reply to comment #11)
> I think this is the same issue that I'm experiencing with another IMAP server
> (where the bug has different manifestation).
> See: https://bugzilla.mozilla.org/show_bug.cgi?id=532360
> I can confirm that Gmail folders have no newlines between emails.
According to comment by bug opener of bug 532360, null line(after last message header) is added in Tb's offline-store file if "offline use" is enabled, even though original mail downloaded from IMAP server doesn't have null line after last message header.
And, you apparantly enables "offline use" because you are talking about offline-store file for "[Gmail]/All Mail".
What do you mean by the "Gmail folders"?
How did you confirm "Gmail folders have no newlines between emails"?
Do you currently disable "offline use" of "[Gmail]/All Mail", after file size of offline-store file for "[Gmail]/All Mail" reached 38GB?
Reporter | ||
Comment 13•15 years ago
|
||
If I don't select offline use, the offline file is not created at all. But if I do, there are no newlines between emails.
Comment 14•15 years ago
|
||
(In reply to comment #13)
> If I don't select offline use, the offline file is not created at all.
> But if I do, there are no newlines between emails.
Can you attach offline-store file?
1. Create an IMAP folder(say xxx), delete file of xxx if exists,
enable "offline use".
2. Copy some mails which produce "no newlines between emails" to the folder.
3. Open the folder, and wait for download by auto-sync.
4. Click other folder(close folder of xxx).
5. Keep back up of file named xxx.msf and xxx, and attach files to this bug.
Reporter | ||
Comment 15•15 years ago
|
||
3 emails sent by Thunderbird and manually moved to the folder (this is not Gmail account, I can't create a new folder with Gmail easily, but the problem is just the same).
Updated•15 years ago
|
Attachment #417994 -
Attachment mime type: application/octet-stream → text/plain
Comment 16•15 years ago
|
||
(In reply to comment #15)
> Folder with 3 emails
Data in the file({LF}==0x0A).
>(snip)
> {last-maile-data-line-of-previouse-mail}{LF}
> From - Wed Dec 16 22:25:00 2009{LF}
> X-Mozilla-Status: 0001{LF}
> X-Mozilla-Status2: 00000000{LF}
> Return-Path: <SimonT@mail.muni.cz>{LF}
>(snip)
(i) All data lines ends with {LF} (0x0A, instead of {CRLF}==0x0D0A).
(ii) "From " line of Unix Mbox is same as one in local mal folder file.
(iii) X-Mozilla-Status:/X-Mozilla-Status2: is written.
How did you get the data? Copied mails to local mail folder?
"new-line of OS" is used for offline-store file of IMAP folder?
Reporter | ||
Comment 17•15 years ago
|
||
Copied to IMAP offline folder.
Reporter | ||
Comment 18•15 years ago
|
||
Reporter | ||
Comment 19•15 years ago
|
||
+ I always do a compact&reindex
Updated•15 years ago
|
Attachment #418010 -
Attachment mime type: application/octet-stream → text/plain
Comment 20•15 years ago
|
||
Although "new line" caharacter is LF instead of CRLF, all mails you attached has null line after message headers(LF only line after last message header).
bug 532360 is irrelevant to mails you attached.
Reporter | ||
Comment 21•15 years ago
|
||
Are you sure? What I understand, the bug 532360 came to the conclusion that Thunderbird can't parse this format (missing newlines between emails).
This can be easily the cause of this bug. Thunderbird cannot parse the mailbox he stored and therefore he can't compact it.
Btw. I have an update, new profile now created a correctly sized file (I'm continuously deleting and receiving emails on the account, so I probably deleted an email that was causing this). Its with the same version of Thunderbird.
Comment 22•15 years ago
|
||
(In reply to comment #21)
> Are you sure? What I understand, the bug 532360 came to the conclusion that
> Thunderbird can't parse this format (missing newlines between emails).
Issue found in Bug 532360 :
If mail has no mail body and no null line after message headers(i.e. header
only mail), Tb requires null line after message header to process mail
properly. If mail in unix mbox file(local folder file, offline-store file),
the null line(==null line just before "From " line) is inserted by Tb.
So, problem of bug 532360 happens on IMAP folder of no "offline use".
Reporter | ||
Comment 23•15 years ago
|
||
The other bug manifests itself in both cases (offline/no-offline). Thunderbird only adds the null line ONLY if the mail is moved to a local folder.
Comment 24•15 years ago
|
||
wada, no fixes for this in 3.0.1?
Reporter | ||
Comment 25•15 years ago
|
||
(In reply to comment #24)
> wada, no fixes for this in 3.0.1?
I think I erased the mail that caused the problems, the issue disappeared before I upgraded thunderbird (simply by recreating the account, which didn't work before).
Comment 26•15 years ago
|
||
For original problem of this bug.
(In reply to comment #0)
> Gecko/20091122 SUSE/3.0.0-4.1 Thunderbird/3.0
> I have 280MB of emails in Gmail, but the All Mail file has 38GB (after compacting).
(In reply to comment #10)
> I'm talking about file size of single files.
Probably Bug 537498(fixed by bug 532323). Because offset value is corrupted, compact probably doesn't help. Even if compact is successful, data in offline-store is corrupted, because data is copied from wrong offset due to wrap around of offset value.
> Bug 537498 If IMAP offline-store file size exceeds 4GB, mails downloaded
> at over 4GB can not be read, and downloaded again & again, even if mail
> folder size is within 4GB (4GB limit is on Win, 2GB limit if Linux/Mac)
As Bug 487992 is alreay fixed, rebuild-index is a recovery procedure.
> Bug 487992 File size of offline store of IMAP folder increases upon each "Rebuild Index"
Closing as DUP of bug 537498.
If wrong, re-open bug, please, with attaching required data for diagnosis.
Status: UNCONFIRMED → RESOLVED
Closed: 15 years ago
Resolution: --- → DUPLICATE
You need to log in
before you can comment on or make changes to this bug.
Description
•