Closed Bug 450246 Opened 16 years ago Closed 15 years ago

Removed tag(keyword of IMAP) of IMAP mail by other client is not reflected(doesn't disappear), until Rebuild Index or new addition of other tag

Categories

(MailNews Core :: Networking: IMAP, defect)

Product:
MailNews Core
Component:
Networking: IMAP
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
minor

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: World, Assigned: Bienvenu)

References

(Blocks 1 open bug)

Details

(Whiteboard: WFM with Tb3.0. bug 518581/693204 has similar phenomenon but different problem) 

Removed tag(keyword of IMAP) of IMAP mail by other client is not reflected(doesn't disappear), until Rebuild Index or new addition of other tag.

Problem was observed with Tb 2.0.0.14, Tb Trunk(2008/8/09 build),Seamonkey 1.1.11 on MS Win-XP SP3.
(0) Define IMAP account,tag-1,tag-2,tag-3 with Seamonkey 1.1.11 and Thunderbird
(1) At client-1(Seamonkey 1.1.11 or Thunderbird) 
    At folde-A: Add tag-1, tag-2 to mail-A-1
(2) At Client-2(Thunderbird or Seamonkey 1.1.1)
    Open folder-A: mail-A-1 has tag-1, tag-2
(3) At client-1
    At folde-A: Remove tag-1 from mail-A-1
(4) At client-2
    Re-open folder-A(click other folder then click folder-A again)
    ==> Both tag-1 & tag-2 is still displayed
    Re-build Index => tag-1 disappears.
(5) At client-1
    At folde-A: Add tag-3 to mail-A-1 (currently tag-2,tag-3)
(6) At client-2
    Re-open folder-A(click other folder then click folder-A again)
    ==> Lack of tag-1 is refleted, and tag-3 is added => tag-2,tag-3

At step (4), IMAP server returns tag-2 only for FETCH. Tb,Sm seems to ignore lack of tag-1 until newly added tag-X is detected.

Note:
When Gmail IMAP, above problem is easily generated within single client, because of special design/spec of Gmail IMAP. Main purpose of this bug is to track funny phenomenon on Tb/bug of Tb when Tb meets special Gmail/Gmail IMAP behaviour.
(1) Copy a mail(mail-1) from folder-A(mail-1-A) to folder-B(mail-1-B)
    Because Gmail holds single instance of mail data only, they are same mail.
(2) At folder-A: add tag-1,tag-2 to mail-1-A
(3) Open folder-B: tag-1,tag-2 is set on mail-1-B
(4) At folder-A: remove tag-1 from mail-1-A
    Because of single instance of mail data, tag-1 is removed from all "copy"
    of the mail in other Gmail IMAP mail folder.
(5) Re-open folder-B: tag-1,tag-2 are still set on mail-1-B
    Because of single instance of mail data, FETCH says no tag-1.
(6-a) At folder-B: Rebuild Index => tag-1 of mail-1-B disappears
      (tag-1 is already removed at step (4))
(6-b) At folder-A: add tag-3 to mail-1-A
      Re-open folder-B: tag-1 is removed, tag-3 is added(mail1-B: tag-2,tag-3)
Product: Core → MailNews Core
Blocks: tb-tagsmeta
Bug 224217 is found for same symptom on "Label" with Tb 1/1.5. It looks old problem - remove of flag(keyword) at IMAP server is ignored until addition of new keyword. It's possibly a required behavior to support "local only flag(keyword/tag)" for IMAP server who doesn't support user keyword. If tag data is always removed from MailDb when "no flag(keyword) in IMAP response", "local only keyword(tag) support" won't work. Improvement is possible, but what can Tb do in (Case-B-2)?
  (Case-A) Other flag(keyword) exists in response -> Remove from MailDB
           (Test case of this bug)
  (Case-B) Last flag(keyword) is removed from a mail (no keyword in response)
           (Bug 224217's case)
  (Case-B-1) If flag(keyword) exists in other mails, remove from MailDB
  (Case-B-2) If flag(keyword) doesn't exist in any other mail,
             should Tb remove from mailDB? Or should keep in MailDB?

How can IMAP client distinguish "IMAP server without user keyword support" from "IMAP server with user keyword support"?
Note: RFC 3502 says.
> http://www.faqs.org/rfcs/rfc3501.html
>         OK [PERMANENTFLAGS (<list of flags>)]
>                     A list of message flags that the client can change
>                     permanently.  If this is missing, the client should
>                     assume that all flags can be changed permanently.
Just an idea:
 - per IMAP account pref of local_only_tag_support=true/false
 - "local only tag(flag/keyword in IMAP)" is supported only when the pref is true
I do have the same problem but I do not understand WADAs reasoning. For me it's just a bug. When I set a flag it is reflected on the IMAP server, when I delete it this information is not reflected. This should be easy to test and very simple to fix. Also, I do not understand the Importance "minor" setting. Yes, the bug does not affect the core behaviour of receiving and sending email but it renders the "flags" feature near to useless for all IMAP users.
we can tell by the flags returned from the select response whether the server supports user-defined keywords or not.
(In reply to comment #4)
\* in PERMANENTFLAGS?
(Gmail IMAP example)
> select "Drafts" 
> * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)]
Yes, that's what \* in PERMANENTFLAGS means. I can't tell if Markus/WADA has tried to reproduce this in 3.0 beta 1
> I can't tell if Markus/WADA has tried to reproduce this in 3.0 beta 1

Tested again with Tb latest trunk, using Gmail IMAP's "[Gmail]/All Mail" folder.
> Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b3pre) Gecko/20090216 Shredder/3.0b2pre

(1) Seamonkey : Add tag-001, tag-002, tag-003 to UID=4220 & UID=4221
(2) Tb trunk  : Rebuild Index, UID=4220 & UID=4221 has tag-001, tag-002, tag-003
(3) Seamonkey : Remove tag-002 from UID=4220 & UID=4221
(4) Tb trunk  : Nothing happens (No notify via IDLE. FETCH is not issued by Tb)
(5) Tb trunk  : Collapse account, expand account
    => Following flow occurred, and tag-002 of UID=4221 was removed by Tb.
       tag-002 of UID=4220 was not removed because no information from server. 

> 00000020  0.82336581 [1996] 2932[59fc380]: 30b9400:imap.gmail.com:S-Test-002:SendData: 22 IDLE  
> 00000022  1.02151310 [1996] 2932[59fc380]: 30b9400:imap.gmail.com:S-Test-002:CreateNewLineFromSocket: + idling  
> 00000023 76.07429504 [1996] 4540[31a4780]: 5eb9400:imap.gmail.com:S-[Gmail]/All Mail:SendData: DONE  
> 00000025 76.26722717 [1996] 4540[31a4780]: 5eb9400:imap.gmail.com:S-[Gmail]/All Mail:CreateNewLineFromSocket: 26 OK IDLE terminated (Success)  
> 00000028 76.26807404 [1996] 4540[31a4780]: 5eb9400:imap.gmail.com:S-[Gmail]/All Mail:SendData: 27 check  
> 00000030 76.45956421 [1996] 4540[31a4780]: 5eb9400:imap.gmail.com:S-[Gmail]/All Mail:CreateNewLineFromSocket: 27 OK Success  
> 00000031 76.45980072 [1996] 4540[31a4780]: 5eb9400:imap.gmail.com:S-[Gmail]/All Mail:SendData: 28 getquotaroot "[Gmail]/All Mail"  
> 00000033 76.65528107 [1996] 4540[31a4780]: 5eb9400:imap.gmail.com:S-[Gmail]/All Mail:CreateNewLineFromSocket: * QUOTAROOT "[Gmail]/All Mail" ""  
> 00000035 76.65543365 [1996] 4540[31a4780]: 5eb9400:imap.gmail.com:S-[Gmail]/All Mail:CreateNewLineFromSocket: * QUOTA "" (STORAGE 4209 7471200)  
> 00000037 76.65570831 [1996] 4540[31a4780]: 5eb9400:imap.gmail.com:S-[Gmail]/All Mail:CreateNewLineFromSocket: 28 OK Success  
> 00000038 76.65579987 [1996] 4540[31a4780]: 5eb9400:imap.gmail.com:S-[Gmail]/All Mail:SendData: 29 UID fetch 4222:* (FLAGS)  
> 00000040 76.85249329 [1996] 4540[31a4780]: 5eb9400:imap.gmail.com:S-[Gmail]/All Mail:CreateNewLineFromSocket: * 169 FETCH (UID 4221 FLAGS (\Seen tag-003 tag-001))  
> 00000042 76.85267639 [1996] 4540[31a4780]: 5eb9400:imap.gmail.com:S-[Gmail]/All Mail:CreateNewLineFromSocket: 29 OK Success  
> 00000043 77.05358887 [1996] 4540[31a4780]: 5eb9400:imap.gmail.com:S-[Gmail]/All Mail:SendData: 30 IDLE  
> 00000045 77.24679565 [1996] 4540[31a4780]: 5eb9400:imap.gmail.com:S-[Gmail]/All Mail:CreateNewLineFromSocket: + idling

(6) Tb trunk  : After some operations(I did "restart of Tb" too),
                tag-002 of UID=4220 was also removed automatically by Tb.

Problem looks to be already resolved.

> 29 UID fetch 4222:* (FLAGS)  
> * 169 FETCH (UID 4221 FLAGS (\Seen tag-003 tag-001))  
> 29 OK Success

"No response for UID=4220 is bug of Gmail IMAP? Or simply delay in Gmail/Gmail IMAP? Or "UID fetch 4222:* (FLAGS)" by Tb is not appropriate?
Gmail IMAP currently has following issue.
> http://mail.google.com/support/bin/answer.py?answer=96926&topic=12922
> Updates not reflected in multiple clients
This bug is WORKSFORME for me because Tb removed tag-002 when next response was received.
> * 169 FETCH (UID 4221 FLAGS (\Seen tag-003 tag-001))  

To Markus Meyer:
Can you reproduce problem with latest-trunk?
> Can you reproduce problem with latest-trunk?

Actually I'd rather not because these are my two work computers and I'm a bit scared of switching them both to a nightly build...

However, I will test it as soon as a new release comes out and report back. In the mean time, thanks for trying to fix it.
(In reply to comment #9)
> I'm a bit scared of switching them both to a nightly build...

Don't switch to nightly build. Test with new profile please, in order to avoid corruption of daily-use-profile.
1. Decompress nightly build. If MS Win, download ZIP build and unzip it.
> http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-trunk/
2. Create new profile, define IMAP account(s)
3. Disable(uncheck) next option, which is for feature under construction.
> Tools/Options/Advanced/General
>   Advanced Configuration
>     - Enable Global Search and Indexer
> Account Settings/Syncing & Disk Space
>   Message Synchronizing 
>     - Keep message for this account on this computer
4. If test of POP3 with nightly is required, define POP3 accounts with "Leave Messages on Server", in order to avoid DLET of downloaded mails.
re: [comment 7]

see also bug bug 402793 # 22

gmail's emulation of IMAP as it relates to FLAGS leaves much to be desired.

Does tb function as expected with a real-IMAP server?  if joy keep sanity.  (gmail's emulation of IMAP is NOT a flavor of IMAP)
This bugs appears to be fixed in Thunderbird 3.0.1
If other than Inbox of Gmail IMAP, I also could't see this bug with Tb 3.0.3. Tag removed by other client disappered by folder re-open(click other folder, click the folder again).
However, if Inbox of Gmail IMAP, situation was worse than before.
  - Tag removed by other client stays.
  - Tag removed by other client stays even after addition of tag.
 - Tag removed/added at othe Gmail IMAP folder than Inbox
   is not reflected to Inbox of same Gmail IMAP account.
Problem was observed on any Gmail IMAP folder, and was seen on \Seen flag too.
And, add/remove of flag by "other client" was not mandatory.
Problem occurs by add/remove of flag of a mail in other Gmail IMAP folder by a single instance of Tb, even when the single instance of Tb only is connection to Gmail.
"Work offline, then Work online, re-open of mail folder" was required to force re-synchronization of already downloaded mail, if problem occurred.
Current problem with Tb 3 can be said as next?
  If folder is already opened, re-synchronization of flag of
  already downloaded mail is not executed.
Problem I saw in Comment #13 to Comment #15 was bug 518581.

If "max cached connections=5" is set, the folder can be selected at a cached connection. In this case, next command is issued upon folder click.
> (UID=11668 is next to highest UID)
> S-INBOX:SendData: 33 UID fetch 11668:* (FLAGS)
> S-INBOX:CreateNewLineFromSocket: * 4 FETCH (UID 11667 FLAGS (NonJunk \Seen)) 
> S-INBOX:CreateNewLineFromSocket: 33 OK Success

If "max cached connections=1" is set, folder switch occurs at only one connection, then select followed by "uid 1:* fetch (flags)" is issued.  
> S-A-00/A-03:SendData: 127 select "INBOX" 
> S-INBOX:CreateNewLineFromSocket: * FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
> (snip)
> S-INBOX:CreateNewLineFromSocket: 127 OK [READ-WRITE] INBOX selected. (Success)
> S-INBOX:SendData: 128 getquotaroot "INBOX"
> (snip)
> S-INBOX:CreateNewLineFromSocket: 128 OK Success 
> S-INBOX:SendData: 129 UID fetch 1:* (FLAGS)

Original problem of this bug was that "removed tag(flag) by other client" was not removed even when "uid 1:* fetch (flags)" was issued.

WORKSFORME with Tb 3.0.
See bug_518581 for similar phenomenon(tag doesn't appear/disappear) but different problem("uid 1:* fetch (flags)" is not issued until "select after folder switch happens").
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → WORKSFORME
Whiteboard: WORKSFORME_with_Tb3.0 See_bug_518581_for_similar_phenomenon_but_different_problem
(whiteboard needs spaces so it can wrap in bug queries. Note bug 518581 got duped to bug 693204 )
Whiteboard: WORKSFORME_with_Tb3.0 See_bug_518581_for_similar_phenomenon_but_different_problem → WFM with Tb3.0. bug 518581/693204 has similar phenomenon but different problem
You need to log in before you can comment on or make changes to this bug.