Closed
Bug 1054232
Opened 10 years ago
Closed 9 years ago
On Thunderbird 31 downloading duplicates e-mail in pop account
Categories
(MailNews Core :: Networking: POP, defect)
Tracking
(Not tracked)
RESOLVED
WORKSFORME
People
(Reporter: ajolivethabiby, Unassigned)
References
Details
(Keywords: regression, Whiteboard: [regression:TB26][needs protocol log])
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.78.2 (KHTML, like Gecko) Version/7.0.6 Safari/537.78.2
Steps to reproduce:
Tried Safe Mode. Deleted the index. Shut and restarted multiple times. Looked on the webmail version to see if the e-mails were duplicated there (they were not).
Actual results:
Nothing changed.
Expected results:
Duplicate downloads should have stopped.
Summary: When I updated Thunderbird to 31, the downloading of e-mails has gone berserk - every two minutes, I get a new version. Only way to stop is to close Thunderbird. → When I updated Thunderbird to 31, the downloading of e-mails has gone berserk - every two minutes, I get a duplicate of the same e-mail. Only way to stop is to close Thunderbird.
Summary: When I updated Thunderbird to 31, the downloading of e-mails has gone berserk - every two minutes, I get a duplicate of the same e-mail. Only way to stop is to close Thunderbird. → Since I updated Thunderbird to 31 yesterday afternoon, the downloading of e-mails has gone berserk - every two minutes, I get a duplicate of the same e-mail. Only way to stop is to close Thunderbird.
Comment 1•10 years ago
|
||
(this doesn't block development or release, so not blocker serverity)
What happens when you start thunderbird in safe mode?
https://support.mozilla.org/en-US/kb/safe-mode
Severity: blocker → major
Flags: needinfo?(ajolivethabiby)
Summary: Since I updated Thunderbird to 31 yesterday afternoon, the downloading of e-mails has gone berserk - every two minutes, I get a duplicate of the same e-mail. Only way to stop is to close Thunderbird. → On Thunderbird 31 downloading of e-mails has gone berserk - every two minutes, I get a duplicate of the same e-mail.
I think it is extremely serious: if this continues more than one or two days, I will have to stop using Thunderbird.
I had tried safe mode yesterday, and it made no difference.
(In reply to Wayne Mery (:wsmwk) from comment #1)
> (this doesn't block development or release, so not blocker serverity)
>
> What happens when you start thunderbird in safe mode?
> https://support.mozilla.org/en-US/kb/safe-mode
Flags: needinfo?(ajolivethabiby)
Comment 3•10 years ago
|
||
no one else is reporting this as far as I know, so there's not enough information here to address the issue.
Is the inbox pop or imap?
What is size of inbox? (right click on folder and pick properties)
What addons are listed in Help | Troubleshooting?
Flags: needinfo?(ajolivethabiby)
All inboxes: POP3
Here is the information from the Troubleshooter:
Application Basics
Name: Thunderbird
Version: 31.0
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Thunderbird/31.0
Profile Folder: Show in Finder
Application Build ID: 20140717165725
Enabled Plugins: about:plugins
Build Configuration: about:buildconfig
Memory Use: about:memory
Mail and News Accounts
account1:
INCOMING: account1, , (none) Local Folders, plain, passwordCleartext
account2:
INCOMING: account2, , (pop3) mail.samuelsagepartners.com:110, plain, passwordCleartext
OUTGOING: smtp.samuelsagepartners.com:587, plain, passwordCleartext, true
account3:
INCOMING: account3, , (pop3) pop3.live.com:995, SSL, passwordCleartext
OUTGOING: smtp.live.com:587, alwaysSTARTTLS, passwordCleartext, true
account5:
INCOMING: account5, , (pop3) pop3.live.com:995, SSL, passwordCleartext
OUTGOING: smtp.live.com:587, alwaysSTARTTLS, passwordCleartext, true
account6:
INCOMING: account6, , (pop3) pop3.ortrans.org:110, plain, passwordCleartext
OUTGOING: smtp.ortrans.org:465, SSL, passwordCleartext, true
account7:
INCOMING: account7, , (pop3) pop3.live.com:995, SSL, passwordCleartext
OUTGOING: smtp.live.com:587, alwaysSTARTTLS, passwordCleartext, true
account8:
INCOMING: account8, , (pop3) pop.googlemail.com:995, SSL, passwordCleartext
OUTGOING: smtp.googlemail.com:587, alwaysSTARTTLS, passwordCleartext, true
Crash Reports
Extensions
Dictionnaire français «Classique», 4.3, true, fr-classique@dictionaries.addons.mozilla.org
Gather the Senders !, 1.2.2, true, gatherauthors@fabien-tools.net
LookOut, 1.2.13, true, lookout@aron.rubin
Mail Merge, 3.7.0, true, mailmerge@example.net
Personal Level Indicators, 0.7, true, personal-level-indicators@taval.de
Remove Duplicate Messages, 0.1.13, true, {12345678-1234-1234-1234-123456789abc}
Saved Password Editor, 2.7.1, true, savedpasswordeditor@daniel.dawson
Test Pilot for Thunderbird, 1.3.9, true, tbtestpilot@labs.mozilla.com
ThreadVis, 2.2.0, true, {A23E4120-431F-4753-AE53-5D028C42CFDC}
Thunderbird Captive Portal detector, 0.1, true, jid1-CuEKsiBk2Kg10A@jetpack
ThunderPlunger, 2.14, true, {bfec0750-aa6f-11db-abbd-0800200c9a66}
Important Modified Preferences
Name: Value
accessibility.typeaheadfind.flashBar: 0
browser.cache.disk.capacity: 358400
browser.cache.disk.smart_size_cached_value: 358400
browser.cache.disk.smart_size.first_run: false
browser.cache.disk.smart_size.use_old_max: false
extensions.lastAppVersion: 31.0
font.internaluseonly.changed: false
font.name.monospace.x-western: Comic Sans MS
font.name.sans-serif.x-western: Arial
font.name.serif.x-western: Arial
mail.openMessageBehavior.version: 1
mail.spotlight.enable: true
mail.spotlight.firstRunDone: true
mail.spotlight.global_reindex_time: 1273962071
mailnews.database.global.datastore.id: 6805f92a-c5e0-d345-b414-de1c4df6988
network.cookie.prefsMigrated: true
places.database.lastMaintenance: 1388755159
places.history.expiration.transient_current_max_pages: 100664
places.history.expiration.transient_optimal_database_size: 75161926
plugin.importedState: true
plugin.state.scorch: 0
print.macosx.pagesetup-2: PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz4KPCFET0NUWVBFIHBsaXN0IFBVQkxJQyAiLS8vQXBwbGUvL0RURCBQTElTVCAxLjAvL0VO…
privacy.donottrackheader.enabled: true
Graphics
Vendor ID: 0x10de
Device ID: 0x 863
WebGL Renderer: false
GPU Accelerated Windows: 1/1 OpenGL
AzureCanvasBackend: quartz
AzureSkiaAccelerated: 0
AzureFallbackCanvasBackend: none
AzureContentBackend: quartz
JavaScript
Incremental GC: 1
Accessibility
Activated: 0
Prevent Accessibility: 0
Library Versions
Expected minimum version
Version in use
NSPR
4.10.6
4.10.6
NSS
3.16.2 Basic ECC
3.16.2 Basic ECC
NSS Util
3.16.2
3.16.2
NSS SSL
3.16.2 Basic ECC
3.16.2 Basic ECC
NSS S/MIME
3.16.2 Basic ECC
3.16.2 Basic ECC
Flags: needinfo?(ajolivethabiby)
As for the size of the inboxes :
1 - 357 MB
2 - 166 MB
3 - 14.6 MB
4 - 7.9 MB
5 - 14.6 MB
6 - 357 MB
The problem is the same regardless of the size of the inbox.
Comment 6•10 years ago
|
||
Are those numbers the "size on disk" from properties?
Is the duplicate only in one Inbox?
And always the same message?
Yes, they are from the "size on disk" in properties.
The duplicates are in all inboxes where I have received new mail since installing the new version of Thunderbird.
There is no message other than the information that I have just received new messages from the different senders. Thunderbird doesn't see it as an error - it's as if it doesn't recognize that it has already downloaded the message.
I'm on my webmail, and have verified that there is only one copy of the mails showing on the web servers.
Comment 9•10 years ago
|
||
Even though it still failed in safe mode, my first inclination is this is caused by an addon. So as a first step I recommend you delete (not disable) Remove Duplicate Messages, and thunderplunger, lookout. If it doesn't help you can add them back.
Reporter | ||
Comment 10•10 years ago
|
||
OK - how do I delete them?
Comment 11•10 years ago
|
||
Tools, addons. Remove
Reporter | ||
Comment 12•10 years ago
|
||
OK - done. There are a number of add ons that are waiting for installation (they tell me to restart Thunderbird in order to install - I cannot remove these). Undo doesn't work, either, for them - how can I stop them?
Reporter | ||
Comment 13•10 years ago
|
||
Still the same problem with downloading messages.
Comment 14•10 years ago
|
||
Thanks for the info. The good news is not much has changed in the pop code since version 24.
So we (you) take a couple stabs to find which versions still work (and from that we find the first version that fails). Note - this is not a cop out, but the best path toward getting a solution. Since you filed the bug and can reproduce you are our best hope. Pretty quick - maybe 30 minutes of time.
So
- backup your thunderbird profile directory
- please install https://ftp.mozilla.org/pub/mozilla.org/thunderbird/releases/30.0b1/mac/en-US/
- if 30.0b1 fails in the same way, please install https://ftp.mozilla.org/pub/mozilla.org/thunderbird/releases/27.0b1/mac/en-US/
- if 27.0b1 fails in the same way, please install version 24
- if 24.0.6 also fails (and presuming it previously worked for you), then the problem is either an addon, server or corrupted popstate.dat file
On that thought, what is the size of your popstate.dat file?
Comment 15•10 years ago
|
||
so I don't forget, the two pop checkins of the past year are bug 86233 and bug 760289
Reporter | ||
Comment 16•10 years ago
|
||
Is popstate.dat a mac type file?
OK - many, many thanks for all of your help!! I will try this and let you know
Reporter | ||
Comment 17•10 years ago
|
||
popstate.dat = 680 kb
Reporter | ||
Comment 18•10 years ago
|
||
Hi Wayne - with 24.6.0, the download duplication has stopped. The versions 30.0b1 and 27.0b1 had the same problem as the TB 31.
Thank you again for all of your help - please let me know if you find out what the problem was.
Reporter | ||
Comment 19•10 years ago
|
||
Hi Wayne - with 24.6.0, the download duplication has stopped. The versions 30.0b1 and 27.0b1 had the same problem as the TB 31.
Thank you again for all of your help - please let me know if you find out what the problem was.
Comment 20•10 years ago
|
||
Do you have 'Leave Messages on Server' enabled in your pop account settings??
I'm betting you will find that https://ftp.mozilla.org/pub/mozilla.org/thunderbird/releases/25.0b1/mac/en-US/ works and https://ftp.mozilla.org/pub/mozilla.org/thunderbird/releases/26.0b1/mac/en-US/ fails. If so, then the problem is likely bug 239455, which I discounted because it was designed to avoid the type of problem you are seeing.
Can you try those two beta versions for us?
Component: Untriaged → Networking: POP
Flags: needinfo?(ajolivethabiby)
Product: Thunderbird → MailNews Core
Summary: On Thunderbird 31 downloading of e-mails has gone berserk - every two minutes, I get a duplicate of the same e-mail. → On Thunderbird 31 downloading of e-mails has gone berserk - every two minutes, I get a duplicate of the same e-mail in pop account
Updated•10 years ago
|
Keywords: regression
Reporter | ||
Comment 21•10 years ago
|
||
Hi Wayne -
As you predicted, 25.0b1 is fine, while 26.0b1 shows the same problem as in 31.
All of the e-mail accounts have "leave mail on server" checked.
I hope this helps understand - and thank you again.
Flags: needinfo?(ajolivethabiby)
Comment 22•10 years ago
|
||
Thanks Audrey - that's great detective work! I wish every bug filer would do as well.
I'm tempted to mark this tracking esr because I'm concerned we will get many reports after we unthrottle TB31 updates. But today I find other user reports of this problem, so perhaps Audrey is hitting an edge case.
Johannes (and others) any thoughts on why this is happening?
Assignee: nobody → buchner.johannes
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(buchner.johannes)
Summary: On Thunderbird 31 downloading of e-mails has gone berserk - every two minutes, I get a duplicate of the same e-mail in pop account → On Thunderbird 31 downloading duplicates e-mail in pop account
Whiteboard: [regression:TB26]
Comment 23•10 years ago
|
||
Audrey, is there anything particular about your file-system setup?
How much disk-space is left on the file-system where Thunderbird could attempt to write (in particular the profile folder)?
Have you set up any soft-links that might confuse things (If you do not understand this question, the answer is no)?
Flags: needinfo?(buchner.johannes)
Reporter | ||
Comment 24•10 years ago
|
||
Hi Johannes -
"is there anything particular about your file-system setup?"
Not that I'm aware of. The file system is on SSD. Trim is enabled. I have Mac OS/X 10.9.4.
"How much disk-space is left on the file-system where Thunderbird could attempt to write (in particular the profile folder)?"
54 GB available (out of 480).
"Have you set up any soft-links that might confuse things (If you do not understand this question, the answer is no)?"
Not that I'm aware of.
Please let me know if you need further information.
Kind regards,
Audrey
Comment 25•10 years ago
|
||
Thunderbird for some reason forgets which emails have already been downloaded. So much is clear. The problem could be either that the emails look different to Thunderbird, or that it was not successful in storing its memory. For the latter, I can only say that the patch I contributed was due to Thunderbird not being able to write to a file (popstate.dat in particular), due to a full disk. It does so by creating a new temporary file and tries to move it over. Perhaps this fails for you. If you want to get your hands really dirty, you could use "strace thunderbird 2>&1|grep -i 'open|error' -5 > strace.log" or so to see if it is a file-opening issue related to writing to the profile. Also be sure to check the Javascript console. Otherwise I am out of ideas.
Are your email accounts all with the same server? Is it a popular one? Some POP servers give out new, random IDs for the emails every time you log in. However, this cause seems to be ruled out since it is software-version related.
Comment 26•10 years ago
|
||
We could use a pop3 protocol log to diagnose whether it is a problem on the server (serving unique UIDs of the messages each time) or of TB (always requesting even unchanged UIDs).
Please see https://wiki.mozilla.org/MailNews:Logging, then
set NSPR_LOG_MODULES=pop3:5
set NSPR_LOG_FILE=/path/to/file/name
Please only fetch messages from one server when you do the test and the you can remove private data (e.g. bodies of the messages) from the log file.
Reporter | ||
Comment 27•10 years ago
|
||
@ Johannes -
The e-mail accounts are on hotmail / live, gmail and another (hosted by OVH).
I'm not very technical, so I'm not sure what to do exactly with "strace thunderbird 2>&1|grep -i 'open|error' -5 > strace.log", although I would be happy to try if that would help. I'm on the 25.0b1 version of TB.
@:aceman:
Is this the correct command to use in terminal?
"#!/bin/sh
export NSPR_LOG_MODULES=pop3:5
export NSPR_LOG_FILE=/Users/audreyjolivet-habiby/Documents/pop3.log
/Applications/Thunderbird.app/Contents/MacOS/thunderbird-bin &"
I know all of this is clear to you, but it's new to me - it's my first bug report. What does it mean in plain language?
Thanks you both of you, and Wayne, for your help.
Kind regards,
Audrey
Comment 28•10 years ago
|
||
(In reply to Audrey from comment #27)
> @:aceman:
> Is this the correct command to use in terminal?
> "#!/bin/sh
> export NSPR_LOG_MODULES=pop3:5
> export NSPR_LOG_FILE=/Users/audreyjolivet-habiby/Documents/pop3.log
> /Applications/Thunderbird.app/Contents/MacOS/thunderbird-bin &"
>
> I know all of this is clear to you, but it's new to me - it's my first bug
> report. What does it mean in plain language?
I don't know OS X much but this looks correct for a shell script. If you type it directly into the terminal, you probably don't need the "#!/bin/sh" line.
When you produce the log, please disable any mail download at startup (in account settings). Then click Get mail for the chosen account. Fetch mail twice in a row. You say you should get all messages twice. We will then see what the log looks like for these two mail fetches.
Updated•10 years ago
|
Flags: needinfo?(ajolivethabiby)
Updated•10 years ago
|
Whiteboard: [regression:TB26] → [regression:TB26][needs protocol log]
Comment 29•10 years ago
|
||
This is a long standing problem. It has been around since long before v24, but I didn't report it since I couldn't reproduce it. I can verify that it is not related to disk space problems on the Thunderbird profile disk because I have seen it many times and checked to make sure there was generous disk space available many times for POP accounts (all my testing has been on Windows machines). Recently I saw Outlook report that our server was returning an incorrect message count (server query vs actual headers counted) for an IMAP connection. I was wondering if there was a similar query for POP and whether a mismatch could trigger a duplicate message download. I also noted that for the cases I observed personally, it was more likely to occur on a server inbox that had a lot of messages marked as deleted (i.e. not purged yet). I could make the problem go away for a while if I could get the server to compact the mailbox (Compact of Inbox from an IMAP connection does not accomplish compact on the server). The duplicate download is more likely to occur in Thunderbird 31 if I have both a POP profile/account and an IMAP profile/account for the same server account and I browse both in a Thunderbird session. I also suspect that Thunderbird is not committing deletes consistently for POP accounts. A month ago I reconfigured my email header based junk processing rules so that messages identified as junk would be COPIED to the junk folder and then the original deleted. Either on exit or on startup, Thunderbird should delete the messages, but it isn't keeping up. Our server stamps X-UIDL on messages in the inbox.mbx on the server (Imail), and each pair of duplicates has the same X-UIDL. I just now turned off the account settings checkbox for Junk Settings\Move new junk messages and added a "filter after junk classification" rule to COPY msgs identified by Thunderbird as junk to the Junk folder and delete the original downloaded message. This is all to help avoid the duplicate message download problem, or reduce it's severity.
Comment 30•10 years ago
|
||
Also, the duplicate download seems more likely to occur if I leave Thunderbird running overnight and then browse messages using the IMAP connection.
Comment 31•10 years ago
|
||
Accessing the same server via POP3 and IMAP in parallel does not seem safe to me.
Comment 32•10 years ago
|
||
Sorry, I just saw this bug is still assigned to me. I do not know what to do about the bug report.
I can not see how it was created, and how my patch from another bug could have cause this as a regression. If there is any relation at all?
The lack of other users reporting this issue may indicate that it is unrelated -- unless my patch was dropped from of the builds?
Stefan seems to be another problem, although with the same consequences. Perhaps a new bug would help (with reference here). Also, please report one issue at a time -- ideally, reproduceable :) As I said in an earlier comment, the server may do funny things with the IDs it assigns to messages, which could confuse Thunderbird as well.
Sorry to be not more helpful.
Assignee: buchner.johannes → nobody
Comment 33•9 years ago
|
||
To test if bug 239455's checkin is involved the builds to be tested are...
https://archive.mozilla.org/pub/thunderbird/nightly/2013/09/2013-09-05-04-59-33-comm-central/ (should work) and https://archive.mozilla.org/pub/thunderbird/nightly/2013/09/2013-09-11-03-03-41-comm-central/ (should fail). (the other nightly builds adjacent to 2013-09-09 are unfortunately broken)
Comment 34•9 years ago
|
||
reporter Audrey wrote privately " The problem looks like it is finally solved with the latest version."
Status: NEW → RESOLVED
Closed: 9 years ago
Flags: needinfo?(ajolivethabiby)
Resolution: --- → WORKSFORME
You need to log in
before you can comment on or make changes to this bug.
Description
•