Jay Moran (jaym/lazvon) Reporter
	Description • 16 years ago

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.3) Gecko/2008092414 Firefox/3.0.3
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b1pre) Gecko/20081006 Shredder/3.0a3

With Exchange IMAP that also has a Blackberry Enterprise Server (BES) attached to it, there appear to be message duplication problems. At least in the display/local message cache. If you hit the Exchange IMAP server with a different IMAP client there is only a single message on the server, so at least Shredder/Thunderbird isn't duplicating the messages there.

You can even use a differen IMAP client at the same time you are seeing duplicated messages in Thunderbird, and they do not appear. Also when you quit and restart Thunderbird the duplicates are gone (I assume this behavior might be different with Offline Messages enabled, I might try that).

My Inbox will "create" 2, 3, 4 copies of any given message. I hypothesize this might be because the BES servers from what I understand grab a copy of messages out of the Exchange Mail Store and reset the read flag (I think) on them. Also, when a message is read on your blackberry it changes some of the flags for the message.

With Shredder/Thunderbird 3.0 Alpha 2 and previous versions of Thunderbird you would just see the message disappear for a few milliseconds and maybe change from read to unread, but there wasn't duplication. Now, it seems each action that the BES performs creates a "duplicate" of the message in the displayed Inbox.

As an aside, the BES also will take a message that first shows up as "PLAIN" text and thus mono-spaced and changes it to WYSIWIG text using the base Arial type font. More evidence that the BES is "picking" the message up and could be causing some problems.

Reproducible: Sometimes

Steps to Reproduce:
1. Connect Thunderbird 3.0 Alpha 3 to an Exchange IMAP Server that has a Blackberry Enterprise Server running against your mailbox.
2. Wait for incoming mail.
3. What the message get duplicated a few times.
Actual Results:  
Multiple copies of the message appear to be in your Inbox.

Expected Results:  
Single copy of the message is all that should be there.

	Jay Moran (jaym/lazvon) Reporter
	Comment 1 • 16 years ago

Unfortunately we use SSL for our Exchange IMAP. I can see about getting a test account on a test Exchange IMAP system with unencrypted SSL and do a packet capture of that if it will help.

Also, is there a way to do detailed logging of all of the IMAP transactions that I could turn on? Could do that as well to provide some more insight into what is going on.

	Dan Mosedale (:dmosedale, :dmose)
	Comment 2 • 16 years ago

<https://wiki.mozilla.org/MailNews:Logging> has documentation on how to log IMAP (and other) stuff.

	Jay Moran (jaym/lazvon) Reporter
	Comment 3 • 16 years ago

Thanks Dan for the pointer on setting up IMAP logging.

This file shows the issue. I sent two messages from my personal Google Mail for Domains account to my work Exchange Account. You can see the two messages arrive and then when BES picks them up for delivery to my Blackberry, there are suddenly two extra copies (3 total) that Thunderbird displays.

Again, opened up Outlook Web Access and there was only a single message in them. With Alpha2, the messages "flicker" and momentarily disappear but never do they duplicate while BES is picking them up.

BTW, I'm PRETTY sure that it is BES, because as the messages are being duplicated in Thunderbird, my Blackberry starts buzzing with message delivery. The Blackberry only has a single copy of each message as well.

Log file was hopefully completely sanitized. :)

	Wayne Mery (:wsmwk)
	Comment 4 • 16 years ago

I don't know if it's relevant here, but a couple days ago we had a discussion here at work about blackberry - it seems blackberry tends to hammer servers.

	Jay Moran (jaym/lazvon) Reporter
	Comment 5 • 16 years ago

Yes, at work for me we have a rather large mail system... okay, top 3 or 4 in the world in terms of number of users... Blackberry Internet Services (BIS) REALLY, REALLY hammer IMAP/POP servers. WAY, WAY, WAY too many checks so that mail is pushed out very quickly. You really have to pay attention to your optimizations to make sure your system is not severely impacted.

With Blackberry Enterprise Servers (BES) for Exchange, it isn't QUITE as bad from what I can tell. I think it is about once every 15-30 seconds. I don't work on the Enterprise side of the house any more, and never got to play with the BES when I did, but I would think there would be some controls on how often it checks for new messages.

In this particular case, I don't believe the frequency has anything to do with the problem.

Remember, this only reared its head with Alpha 3 for me. Everything is working awesomely in Alpha 2.

I wish I could remember the previous bug from last year that had similar characteristics... I'll search through my previous Bugzilla filings/comments and see if I can find a bug. I might not have said anything though and just stuck to a build that worked.

	Jay Moran (jaym/lazvon) Reporter
	Comment 6 • 16 years ago

I should have known. *grumble*

New Profile fixes this. I'll do a diff and see what settings I have different in the profiles. Meeting time now though... later today I hope.

	Jay Moran (jaym/lazvon) Reporter
	Comment 7 • 16 years ago

Here are the Profile Diffs (at least the ones I felt my have any chance of impact). I had just changed the chunk and fetch config changes, they were all defaults the first time I used Alpha3. I stumbled upon those fixes for 0 byte attachment issues I have every once in a while with Exchange IMAP. Found them while looking through Wayne's search he left for me to look through over on Mozillazine (thanks Wayne, you fixed another of my issues (so far) without even trying!)

< user_pref("mail.imap.chunk_size", 360448);
< user_pref("mail.imap.fetch_by_chunks", false);
< user_pref("mail.imap.min_chunk_size_threshold", 540672);
< user_pref("mail.last_msg_movecopy_target_uri", "mailbox://nobody@Local%20Folders/Archive%20-%20Received");
< user_pref("mail.preferences.advanced.selectedTabIndex", 0);< user_pref("mail.preferences.compose.selectedTabIndex", 0);< user_pref("mail.preferences.display.selectedTabIndex", 0);< user_pref("mail.preferences.privacy.selectedTabIndex", 0);< user_pref("mail.server.default.fetch_by_chunks", false);< user_pref("mail.server.server1.ageLimit", 30);< user_pref("mail.server.server1.cleanupBodies", false);< user_pref("mail.server.server1.daysToKeepBodies", 30);< user_pref("mail.server.server1.daysToKeepHdrs", 30);< user_pref("mail.server.server1.downloadByDate", false);< user_pref("mail.server.server1.downloadUnreadOnly", false);< user_pref("mail.server.server1.empty_trash_on_exit", true);< user_pref("mail.server.server1.keepUnreadOnly", false);< user_pref("mail.server.server1.numHdrsToKeep", 30);< user_pref("mail.server.server1.spamActionTargetAccount", "mailbox://nobody@Local%20Folders");< user_pref("mail.server.server1.spamActionTargetFolder", "mailbox://nobody@Local%20Folders/Junk");< user_pref("mail.server.server2.check_time", 5);> user_pref("mail.server.server2.check_time", 1);< user_pref("mail.server.server2.cleanup_inbox_on_exit", true);< user_pref("mail.server.server2.download_bodies_on_get_new_mail", false);< user_pref("mail.server.server2.empty_trash_on_exit", true);< user_pref("mail.server.server2.moveOnSpam", true);< user_pref("mail.server.server2.moveTargetMode", 1);
< user_pref("mail.server.server2.name", "Work Mail");
> user_pref("mail.server.server2.name", "Work");
< user_pref("mail.server.server2.offline_download", false);
< user_pref("mail.server.server2.purgeSpam", true);
< user_pref("mail.server.server2.purgeSpamInterval", 7);
< user_pref("mail.server.server2.spamActionTargetFolder", "imap://XXX@XXX.com/Junk E-mail");
> user_pref("mail.server.server2.spamActionTargetFolder", "mailbox://nobody@Local%20Folders/Junk");
< user_pref("mail.server.server2.trash_folder_name", "Deleted Items");
< user_pref("mail.server.server2.useServerFilter", true);
< user_pref("mail.smtpserver.smtp1.description", "Work SMTP");
< user_pref("mail.smtpserver.smtp1.try_ssl", 1);
> user_pref("mail.smtpserver.smtp1.try_ssl", 2);
< user_pref("mail.spam.logging.enabled", true);
< user_pref("mail.spam.manualMark", true);
< user_pref("mail.spam.markAsReadOnSpam", true);
< user_pref("mailnews.ui.junk.firstuse", false);
< user_pref("network.negotiate-auth.gsslib", "/dev/null");
< user_pref("network.negotiate-auth.using-native-gsslib", false);
< user_pref("pref.downloads.disable_button.edit_actions", false);

	Jay Moran (jaym/lazvon) Reporter
	Comment 8 • 16 years ago

Nope. After a couple of hours of sitting messages started duplicating. So, fresh profile does NOT fix this. 

Happy to help test other things. But Alpha 3 is NOT happy with Exchange IMAP and Blackberry Enterprise Server.

I can also email a private copy of the INBOX file off the hard drive if that might help.

	gene c
	Comment 9 • 16 years ago

I see the same issue - started happening when I got a blackberry - still happening as of todays build. I typically get 2 or 3 extra messages  - quitting and restarting shredder and inbox looks normal. After a period of time it starts again - once it starts every message is duplicated. 

Only work around I have for now is restarting shredder.

	David Ascher (:davida)
	Comment 10 • 16 years ago

Confirming based on #9.

	Jay Moran (jaym/lazvon) Reporter
	Comment 11 • 16 years ago

Retested with: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b3pre) Gecko/20090123 Shredder/3.0b2pre

I can no longer get this bug to surface. Now you just see the message flicker three times instead of creating three copies when BES picks the message up. That is the expected behavior. So, I'd say this is RESOLVED FIXED, not sure what or when it was fixed though.

Just to verify it wasn't something on my Exchange infrastructure I retested with Shredder Alpha 3 Build 2 (Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b1pre) Gecko/20081006 Shredder/3.0a3) and got the same duplicating message results.

All RESOLVED FIXED now.

	Magnus Melin [:mkmelin]
	Comment 12 • 16 years ago

->WFM as we don't know what patch fixed it.