With the landing of Bug 721316, the nsMsgMessageFlag::Offline of a nsIMsgDBHdr (at least in IMAP) no longer reflects the true offline status of the message. With gmail, it is possible to have the same message in several folders (gmail labels). Only the header in the folder where the message is actually stored will have the ::Offline flag set. The message headers for the same message in other folders have the ::Offline flag cleared. the functions nsImapMailFolder::HasMsgOffline(), nsImapMailFolder::GetOfflineMsgFolder() and nsImapMailFolder::GetOfflineFileStream are used to work with the actual offline message. As a result, many of the uses of nsMsgMessageFlag::Offline are broken because an offline message may still exist even if the flag is not set. I plan on using this as a sort of meta bug to link the individual cases

:jprotz, this probably affects gloda too. Since I've never poked my nose in that code, I'm cc'ing you.

Here is a mxr link that shows usage of nsMsgMessageFlag::Offline https://mxr.mozilla.org/comm-central/search?string=offline&find=&findi=&filter=nsMsgMessageFlags.*[Oo]ffline&hitlimit=&tree=comm-central

(In reply to David Lechner (:dlech) from comment #0) > With the landing of Bug 721316, the nsMsgMessageFlag::Offline of a > nsIMsgDBHdr (at least in IMAP) no longer reflects the true offline status of the message. Did Bug 721316 actualy touch ::Offline flag setting in MsgDBHdr? That bug simply added function which stores X-GM-xxx flag data in MsgDBHdr for future use, didn't it? If I uderstand correctly, "pseudo Virtual Folder by utilizing X-GM-MSID", "Gmail like folder display", "Gmail Label is a kind of Tag in Tb", ..., loke one is not implemented yet. > With gmail, it is possible to have the same message in several > folders (gmail labels). > Only the header in the folder where the message is actually stored will have the ::Offline flag set. You are right. different Gmail Label==absolutely Different IMAP folder, and any mail in different mail folder has different msgDBHdr from any other mail in any IMAP folder. And, ::Offline flag in IMAP folder depends on IMAP folder's offline-use setting and auto-sync status. > The message headers for the same message in other folders have the ::Offline flag cleared. > As a result, many of the uses of nsMsgMessageFlag::Offline are broken > because an offline message may still exist even if the flag is not set. Where did you see it in which Thunderbird build? Following is dump data of msgDBHdr object of next mails in Gmail IMAP folder. Two actual mails in Gmail : X-GM-MSGID=m1, X-GM-MSGID=m2 Both mails has Gmail Label of; Offline-Use-Test/OfflineUseOn Offline-Use-Test/OfflineUseOff IMAP offline-use setting in Tb; Offline-Use-Test/OfflineUseOn : Offline-use = On Offline-Use-Test/OfflineUseOff : Offline-use = Off UID of mail in Tb's IMAP folder; mail of X-GM-MSGID=m1 : in Offline-Use-Test/OfflineUseOn : UID=1 in Offline-Use-Test/OfflineUseOff : UID=3 mail of X-GM-MSGID=m2 : in Offline-Use-Test/OfflineUseOn : UID=2 in Offline-Use-Test/OfflineUseOff : UID=2 These mail are different mails in different folder, then these are absolutely different mails in IMAP, so different msgFolder.flags is set. As for ::Offline flag, it's set according to Offline-use setting of Tb's IMAP folder even when Gmail IMAP. > Extracted msgHdr data of all messages in current View > (1) Mail of messageId = 4F81441B.4070206@ops.dti.ne.jp, mime2DecodedSubject = non-moved-mail > Mbox = imap://yatter.one%40gmail.com@imap.gmail.com/Offline-Use-Test/OfflineUseOn, messageKey = 1, messageOffset = 0, messageSize = 1460, offlineMessageSize = 1546, date = 1333871643000000, dateInSeconds = 1333871643, StringProperty_pendingRemoval = , flags = 129, flag_Detail = { FeedMsg = false, IMAPDeleted = false, MDNReportSent = false, Read = true, Replied = false, Marked = false, Expunged = false, HasRe = false, Elided = false, Offline = true, Watched = false, SenderAuthed = false, Partial = false, Queued = false, Forwarded = false, Priorities = false, New = false, Ignored = false, MDNReportNeeded = false, Template = false, Attachment = false, Labels = false, RuntimeOnly = false }, messageId = 4F81441B.4070206@ops.dti.ne.jp, mime2DecodedSubject = non-moved-mail > Mbox = imap://yatter.one%40gmail.com@imap.gmail.com/Offline-Use-Test/OfflineUseOff, messageKey = 3, messageOffset = 3, messageSize = 1460, offlineMessageSize = 0, date = 1333871643000000, dateInSeconds = 1333871643, StringProperty_pendingRemoval = , flags = 1, flag_Detail = { FeedMsg = false, IMAPDeleted = false, MDNReportSent = false, Read = true, Replied = false, Marked = false, Expunged = false, HasRe = false, Elided = false, Offline = false, Watched = false, SenderAuthed = false, Partial = false, Queued = false, Forwarded = false, Priorities = false, New = false, Ignored = false, MDNReportNeeded = false, Template = false, Attachment = false, Labels = false, RuntimeOnly = false }, messageId = 4F81441B.4070206@ops.dti.ne.jp, mime2DecodedSubject = non-moved-mail > (2) messageId = 4F1E54AB.8040004@gmail.com, mime2DecodedSubject = template-1 > Mbox = imap://yatter.one%40gmail.com@imap.gmail.com/Offline-Use-Test/OfflineUseOn, messageKey = 2, messageOffset = 1546, messageSize = 515, offlineMessageSize = 601, date = 1327387819000000, dateInSeconds = 1327387819, StringProperty_pendingRemoval = , flags = 129, flag_Detail = { FeedMsg = false, IMAPDeleted = false, MDNReportSent = false, Read = true, Replied = false, Marked = false, Expunged = false, HasRe = false, Elided = false, Offline = true, Watched = false, SenderAuthed = false, Partial = false, Queued = false, Forwarded = false, Priorities = false, New = false, Ignored = false, MDNReportNeeded = false, Template = false, Attachment = false, Labels = false, RuntimeOnly = false }, messageId = 4F1E54AB.8040004@gmail.com, mime2DecodedSubject = template-1 > Mbox = imap://yatter.one%40gmail.com@imap.gmail.com/Offline-Use-Test/OfflineUseOff, messageKey = 2, messageOffset = 2, messageSize = 515, offlineMessageSize = 0, date = 1327387819000000, dateInSeconds = 1327387819, StringProperty_pendingRemoval = , flags = 1, flag_Detail = { FeedMsg = false, IMAPDeleted = false, MDNReportSent = false, Read = true, Replied = false, Marked = false, Expunged = false, HasRe = false, Elided = false, Offline = false, Watched = false, SenderAuthed = false, Partial = false, Queued = false, Forwarded = false, Priorities = false, New = false, Ignored = false, MDNReportNeeded = false, Template = false, Attachment = false, Labels = false, RuntimeOnly = false }, messageId = 4F1E54AB.8040004@gmail.com, mime2DecodedSubject = template-1 Following is dump data of StringProperty of msgDBHdr of a message in above. X-GM-MSGID is saved in a StringProperty(may be stored in panecea.dat). > [All_StringProperty] = { flags = 81, statusOfset = 0, sender = soarex soarex <soarex@ops.dti.ne.jp>, recipients = Soarex-ops-dti <soarex@ops.dti.ne.jp>, subject = non-moved-mail, message-id = 4F81441B.4070206@ops.dti.ne.jp, date = 4f81441b, dateReceived = 4f81441b, msgCharSet = ISO-2022-JP, X-GM-MSGID = 1398665791633222725, X-GM-THRID = 1398665791633222725, X-GM-LABELS = "\\Inbox" SentX/C&ZeVnLIqe- SentX/BB Offline-Use-Test/OfflineUseOff, sender_name = 450|soarex soarex, priority = 1, size = 5b4, threadParent = ffffffff, msgThreadId = fffffffe, ProtoThreadFlags = 0, msgOffset = 0, storeToken = 0, offlineMsgSize = 60a, numLines = 17, keywords = , label = 0 } I agree with you on problem what you pointed in following case. - All Gmail Label is considered as "Gmail Label". - Any mail data in Gmail IMAP folder(==Gmail Label) is retrieved via [Gmail]/All Mail uilizing X-GM-MSGID. - Each message in Gmail IMAP folder(==Gmail Label) still has same msgDBHdr structure and data as current. - [Gmail]/All Mail is Ofline-use=Off folder. - If a Gmail IMAP folder(==Gmail Label) is Offline-Use=On folder, mismatch of ::Offline flag happens But, IIRC, it's not implemented yet... IIRC, we now can know "originally same mail or not in Gmail" by bug 721316, but no one can use it usefully yet... Is Gloda already utilizes X-GM-MSGID?

Aha! I've read your bug 854162, and viewed nsImapMailFolder::HasMsgOffline() code. > http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapMailFolder.cpp#9590 > 9590 NS_IMETHODIMP nsImapMailFolder::HasMsgOffline(nsMsgKey msgKey, bool *_retval) nsImapMailFolder::HasMsgOffline calls nsImapMailFolder::GetOfflineMsgFolder. > 9595 nsresult rv = GetOfflineMsgFolder(msgKey, getter_AddRefs(msgFolder)); > http://mxr.mozilla.org/comm-central/source/mailnews/imap/src/nsImapMailFolder.cpp#9602 > 9602 NS_IMETHODIMP nsImapMailFolder::GetOfflineMsgFolder(nsMsgKey msgKey, nsIMsgFolder **aMsgFolder) nsImapMailFolder::GetOfflineMsgFolder checks nsMsgMessageFlags::Offline of passed msgKey first. > 9621 if ((msgFlags & nsMsgMessageFlags::Offline)) > 9622 { > 9623 NS_IF_ADDREF(*aMsgFolder = this); > 9624 return NS_OK; > 9625 } When nsMsgMessageFlags::Offline flag of the passed msgKey is Off(somehow headers are refetched and old data in offline-store is discarded, but re-download of message body to offline-store is not done yet), if isGmail, nsImapMailFolder::GetOfflineMsgFolder checks some special folders specified in X-GM-Label of passed msgKey, and search for mail of same X-GM-MGID(originally same mail in Gmail), and checks nsMsgMessageFlags::Offline flag of the originally same mail. > 9638 if (isGMail) >(snip) > 9642 hdr->GetStringProperty("X-GM-LABELS", getter_Copies(labels)); >(snip) > 9689 hdr->GetStringProperty("X-GM-MSGID", getter_Copies(gmMsgID)); >(snip) > 9697 if ((gmFlags & nsMsgMessageFlags::Offline)) I didn't know that X-GM-MSGID is already utilized. (In reply to David Lechner (:dlech) from comment #0) > With the landing of Bug 721316, the nsMsgMessageFlag::Offline of a > nsIMsgDBHdr (at least in IMAP) no longer reflects the true offline status of the message. Cause-and-effect relationship　is reverse, isn't it? Even when somehow refetch of header happened at an Gmail IMAP Mbox and nsMsgMessageFlag::Offline flag of a msgKey is reset, if Gmail IMAP, by utilizing X-GM-MSGID which is fetched and saved by　Bug 721316, problem of refetch of entire message body(such as bug 816327) can be avoided(or postponed) by calling nsImapMailFolder::HasMsgOffline() instead of "doing direct check of nsMsgMessageFlag::Offline flag of msgHDBdr", as you stated in bug 854162, because nsImapMailFolder::GetOfflineMsgFolder already utilizes X-GM-MSGID.

Or phenomenon of bug 816327 is following? By Bug 721316 landed on Tb 17, fetch of all headers is initiated to fetch X-GM-MSGID etc., then nsMsgMessageFlag::Offline flag is reset, thus fetch body[] is re-executed by auto-sync for any mail of same X-GM-MSGID in different Gmail IMAP folder(Gmail Label). And bug 854162 is for following? Because nsImapMailFolder::GetOfflineMsgFolder already ustilizes X-GM-MSGID, excess fetch body[] of above for any Gmail Label of same X-GM-MSGID can be avoided(or postponed) by change from "checking ::Offline flag of currently processd msgKey" to "calling nsImapMailFolder::GetOfflineMsgFolder()".

(In reply to WADA from comment #3) > Where did you see it in which Thunderbird build? This has been since 17.0. > Is Gloda already utilizes X-GM-MSGID? No, it is not, but it is using the nsMsgMessageFlag::Offline flag in several places. I think it could possibly break things in the same way that autosync is broken. > Cause-and-effect relationship　is reverse, isn't it? I think you are missing one thing here. The nsMsgMessageFlag::Offline is not reset. Rather, it is never set at all. For example if we get a new message in our inbox, the message header is saved and then the message body is saved and the nsMsgMessageFlag::Offline is set for the header that message in the inbox. Then if we look at that same message in All Mail, the header is saved in All Mail, but nsImapMailFolder::GetOfflineMsgFolder() sees that the body is stored in the inbox and so they body from the inbox is used when displaying the message in All Mail. As a result, the nsMsgMessageFlag::Offline flag is never set on the header in All Mail because the body was never downloaded in All Mail. So, the phenomenon is this: Only one message header for each X-GM-MSGID has nsMsgMessageFlag::Offline. The folder of this message is the folder where the body of the message is stored. All other headers in other folders with the same X-GM-MSGID will not have nsMsgMessageFlag::Offline set even though the body of the message is saved in another folder. > Or phenomenon of bug 816327 is following? It is more like this. If nsImapMailFolder::GetOfflineMsgFolder != current folder then nsMsgMessageFlag::Offline == false therefore message is added to queue to be downloaded even though offline copy of message may already exist in another folder. > And bug 854162 is for following? Yes. Really, bug 816327 and bug 854162 are the same bug. Bug 816327 is from the user perspective and is not as well defined and therefore may be related to other problems as well. Bug 854162 is from the developer perspective to address a specific, well defined problem that was identified as a result of researching bug 816327.

(In reply to David Lechner (:dlech) from comment #6) > > Cause-and-effect relationship　is reverse, isn't it? > I think you are missing one thing here. > The nsMsgMessageFlag::Offline is not reset. > Rather, it is never set at all. Confirmed it in Tb 17.0.2. (1) Setup - mail of X-GM-MSGID = m1, held in [Gmail]/All Mail always, unless copied/moved to [Gmail]/Trash or [Gmail]/Spam - To make it simple, all special folders except Inbox is Offline-Use=Off - mail of X-GM-MSGID = m1 has Gmail Label=Inbox In Tb, Inbox is Offline-Use=On => body fetched by auto-sync and saved in offline-store file, ::Offlineuse=On is set by auto-sync (2) Create FolderX in Tb, Offline-Use=On (3) Add Gmail Label of FolderX to mail of X-GM-MSGID = m1 (4) Open FolderX in Tb. New mail is detected, message headers including X-GM-x are fetched, and "gmail-imap-account is up to date" is shown at Activity manager. Mail in FolderX can be viewed even while Work Offline mode, but ::Offlineuse flag is Off, and offline-store file is not created. Phenomenon looks for me; (a) auto-sync calls nsImapMailFolder::GetOfflineMsgFolder, and if nsImapMailFolder::GetOfflineMsgFolder != current folder, auto-sync doesn't initiate "fetch body[]" for FolderX", because mail of same X-GM-MSGID is already held in special folder of Offline-use=On. (b) Viewing of message also calls nsImapMailFolder::GetOfflineMsgFolder, and if nsImapMailFolder::GetOfflineMsgFolder != current folder, obtain mail data for same X-GM-ID in offline-store file of special folder. Because of above change, "direct check of nsMsgMessageFlag::Offline" is useless when Gmail IMAP, as you say. Characteristics of X-GM-LABELS; 1. Tb fetches it upon first header fetch of new mail in a folder. Gmail Label can be added/removed anytime, so X-GM-LABELS is not static. X-GM-LABELS of a mail in a Gmail IMAP folder depends on when message header of the mail was fetched. 2. [Gmail]/All Mail, [Gmail]/Trash, [Gmail]/Spam are not set in X-GM-LABELS, because there are not "Gmail Label". 3. When mail of Gmail Label=X is shown as mail in Gmail IMAP folder=X, "X" is not set in X-GM-LABELS of the mail in folder X. So, following can occur. - If re-sync of old mail happenes, if X-GM-LABEL is old data, auto-sync executes "fetch body[]" even when entire message data is held in offline-store file of other Gmail IMAp folder. Because X-GM-LABELS is changed by Gmail any time, I think usage of X-GM-LABLES should; 1. Get X-GM-MSGID of current msgDBHdr. 2. Get msgDBHdr of mail of same X-GM-MSGID in [Gmail]/All Mail or [Gmail]/Trash or [Gmail]/Spam. 4. Fetch X-GM-LABELS of the mail in [Gmail]/xxx to know latest Gmail Label assignment in Gmail. 5. Search "Offlineuse=On Gmail IMAP folders listed in latest X-GM-LABELS" for "msgDBHdr key of same X-GM-MSGID".

FYI. Utilizing of X-GM-MSGID etc. was introduced by following change. > http://hg.mozilla.org/comm-central/diff/bb45e728a12e/mailnews/imap/src/nsImapMailFolder.cpp Change set of it was for Bug 721316. > http://hg.mozilla.org/comm-central/rev/bb45e728a12e

To Atul Jangra [:atuljangra] (owner of 721316): Is there good way to improve? Any referrer to nsMsgMessageFlag::Offline flag should always use nsImapMailFolder::GetOfflineMsgFolder() if Gmail MAP and has to have logic to switch folder if different folder is returned? (note: when you add comment, "need-info flag is automatically reset)

FYI. MMsgDBHdr info and dump of StringProperty of message of same X-GM-MSGID in Tb 17.0.3 on MS Win. (1) X-GM-MSGID = 1397005237944465774 in [Gmail]/All Mail (Offline-Use=Off) > Mbox = imap://yatter.one%40gmail.com@imap.gmail.com/[Gmail]/All Mail, messageKey = 33190, messageOffset = 33190, messageSize = 457, offlineMessageSize = 0, date = 1332288015000000, dateInSeconds = 1332288015, StringProperty_pendingRemoval = , flags = 1, flag_Detail = { FeedMsg = false, IMAPDeleted = false, MDNReportSent = false, Read = true, Replied = false, Marked = false, Expunged = false, HasRe = false, Elided = false, Offline = false, Watched = false, SenderAuthed = false, Partial = false, Queued = false, Forwarded = false, Priorities = false, New = false, Ignored = false, MDNReportNeeded = false, Template = false, Attachment = false, Labels = false, RuntimeOnly = false }, messageId = 002-4F691A0F.8030609@x.x.x, mime2DecodedSubject = abc-def-ghi-xyz-02 > [All_StringProperty] = { flags = 1, statusOfset = 0, sender = abcDEF pqrSTU <x1@x.x.x>, recipients = xYZ xYZ <z1@x.x.x>, subject = abc-def-ghi-xyz-02, message-id = 002-4F691A0F.8030609@x.x.x, date = 4f691a0f, dateReceived = 4f691a0f, priority = 1, msgCharSet = ISO-2022-JP, size = 1c9, threadParent = ffffffff, msgThreadId = 81a6, ProtoThreadFlags = 0, X-GM-MSGID = 1397005237944465774, X-GM-THRID = 1397005237944465774, X-GM-LABELS = INBOX/Inbox0 Offline-Use-Test/Offline-Use-002 Offline-Use-Test/Offline-Use-001, sender_name = 450|abcDEF pqrSTU, keywords = , label = 0 } (2) X-GM-MSGID = 1397005237944465774 in Inbox (Offline-Use=On) > Mbox = imap://yatter.one%40gmail.com@imap.gmail.com/INBOX, messageKey = 12184, messageOffset = 1546, messageSize = 457, offlineMessageSize = 543, date = 1332288015000000, dateInSeconds = 1332288015, StringProperty_pendingRemoval = , flags = 129, flag_Detail = { FeedMsg = false, IMAPDeleted = false, MDNReportSent = false, Read = true, Replied = false, Marked = false, Expunged = false, HasRe = false, Elided = false, Offline = true, Watched = false, SenderAuthed = false, Partial = false, Queued = false, Forwarded = false, Priorities = false, New = false, Ignored = false, MDNReportNeeded = false, Template = false, Attachment = false, Labels = false, RuntimeOnly = false }, messageId = 002-4F691A0F.8030609@x.x.x, mime2DecodedSubject = abc-def-ghi-xyz-02 > [All_StringProperty] = { flags = 81, statusOfset = 0, sender = abcDEF pqrSTU <x1@x.x.x>, recipients = xYZ xYZ <z1@x.x.x>, subject = abc-def-ghi-xyz-02, message-id = 002-4F691A0F.8030609@x.x.x, date = 4f691a0f, dateReceived = 4f691a0f, msgCharSet = ISO-2022-JP, X-GM-MSGID = 1397005237944465774, X-GM-THRID = 1397005237944465774, X-GM-LABELS = INBOX/Inbox0 Offline-Use-Test/Offline-Use-002 Offline-Use-Test/Offline-Use-001, sender_name = 450|abcDEF pqrSTU, offlineMsgSize = 21f, msgOffset = 60a, storeToken = 1546, priority = 1, size = 1c9, threadParent = ffffffff, msgThreadId = 2f98, ProtoThreadFlags = 0, keywords = , label = 0, numLines = f } (3) X-GM-MSGID = 1397005237944465774 in Offline-Use-Test/Offline-Use-001 (Offline-Use=On) > Mbox = imap://yatter.one%40gmail.com@imap.gmail.com/Offline-Use-Test/Offline-Use-001, messageKey = 6, messageOffset = 0, messageSize = 457, offlineMessageSize = 543, date = 1332288015000000, dateInSeconds = 1332288015, StringProperty_pendingRemoval = , flags = 1, flag_Detail = { FeedMsg = false, IMAPDeleted = false, MDNReportSent = false, Read = true, Replied = false, Marked = false, Expunged = false, HasRe = false, Elided = false, Offline = false, Watched = false, SenderAuthed = false, Partial = false, Queued = false, Forwarded = false, Priorities = false, New = false, Ignored = false, MDNReportNeeded = false, Template = false, Attachment = false, Labels = false, RuntimeOnly = false }, messageId = 002-4F691A0F.8030609@x.x.x, mime2DecodedSubject = abc-def-ghi-xyz-02 > [All_StringProperty] = { flags = 1, statusOfset = 0, sender = abcDEF pqrSTU <x1@x.x.x>, recipients = xYZ xYZ <z1@x.x.x>, subject = abc-def-ghi-xyz-02, message-id = 002-4F691A0F.8030609@x.x.x, date = 4f691a0f, dateReceived = 4f691a0f, priority = 1, msgCharSet = ISO-2022-JP, size = 1c9, threadParent = ffffffff, msgThreadId = 6, ProtoThreadFlags = 0, X-GM-MSGID = 1397005237944465774, X-GM-THRID = 1397005237944465774, X-GM-LABELS = "\\Inbox" INBOX/Inbox0 Offline-Use-Test/Offline-Use-002, sender_name = 450|abcDEF pqrSTU, keywords = , label = 0, msgOffset = 0, storeToken = 0, offlineMsgSize = 21f, numLines = f } (4) X-GM-MSGID = 1397005237944465774 in Offline-Use-Test/Offline-Use-002 (Offline-Use=On) > Mbox = imap://yatter.one%40gmail.com@imap.gmail.com/Offline-Use-Test/Offline-Use-002, messageKey = 2, messageOffset = 0, messageSize = 457, offlineMessageSize = 543, date = 1332288015000000, dateInSeconds = 1332288015, StringProperty_pendingRemoval = , flags = 1, flag_Detail = { FeedMsg = false, IMAPDeleted = false, MDNReportSent = false, Read = true, Replied = false, Marked = false, Expunged = false, HasRe = false, Elided = false, Offline = false, Watched = false, SenderAuthed = false, Partial = false, Queued = false, Forwarded = false, Priorities = false, New = false, Ignored = false, MDNReportNeeded = false, Template = false, Attachment = false, Labels = false, RuntimeOnly = false }, messageId = 002-4F691A0F.8030609@x.x.x, mime2DecodedSubject = abc-def-ghi-xyz-02 > [All_StringProperty] = { flags = 1, statusOfset = 0, sender = abcDEF pqrSTU <x1@x.x.x>, recipients = xYZ xYZ <z1@x.x.x>, subject = abc-def-ghi-xyz-02, message-id = 002-4F691A0F.8030609@x.x.x, date = 4f691a0f, dateReceived = 4f691a0f, priority = 1, msgCharSet = ISO-2022-JP, size = 1c9, threadParent = ffffffff, msgThreadId = 6, ProtoThreadFlags = 0, X-GM-MSGID = 1397005237944465774, X-GM-THRID = 1397005237944465774, X-GM-LABELS = "\\Inbox" INBOX/Inbox0 Offline-Use-Test/Offline-Use-002, sender_name = 450|abcDEF pqrSTU, keywords = , label = 0, msgOffset = 0, storeToken = 0, offlineMsgSize = 21f, numLines = f }

A simple way for Gmail IMAP. - If Gmail IMAP, messageKey==X-GM-MSG-ID instead of messageKey==UID. If required, extend messageKey to 64bits integer. - If non [Gmail]/All Mail, [Gmail]/Trash, [Gmail]/Spam, mail is always accessed using messageKey==X-GM-MSG-ID via [Gmail]/All Mail or [Gmail]/Trash or [Gmail]/Spam. And messageKey==X-GM-MSG-ID is converted to UID, and IMAP command is issued using UID. - is_gmail is set by X-GM-EXT-1 capability response. is_gmail is reset only when X-GM-EXT-1 doesn't exist in actuallly returned capability response. Manual change of is_gmail is intentional corruption of prefs by user. So, change via Confg Editor should be ignored always. Problems around it. - What should we do when user intentionally hides [Gmail]/All Mail or [Gmail]/Trash or [Gmail]/Spam at Gmail's Label setting? - What can we do when user sets Offline-Use=Off for [Gmail]/All Mail or [Gmail]/Trash or [Gmail]/Spam in Tb? "Offline-Use=On at Inbox" is intentional auto-sync request by Tb user. So, I think "fetch body[] by auto-sync of Tb at [Gmail]/All Mail regardles of Offline-Use setting" is reasonable action, if original request is from "Gmail IMAP folder of Offline-Use=On".

A counterplan to "intentional hiding of [Gmail]/All Mail by user". - is_AllMail_shown : If special folder of "AllMail" is returned in XLIST, and if the Mbox specified in XLIST is returned in LIST response, is_AllMail_shown = true. - If(true==is_gmail&&is_true==AllMail_shown), messageKey=X-GM-MSGID mode, Else messageKey=UID mode.

For msgDBHdr change. Because messageKey, messageOffset, etc. is already cleanly separated, additional msgDBHdr.UID and msgDBHdr.messegeKeyMode (default==UID-mode:0x1=on, X-GM-MSGID-mode:0x2=on, ...) like one is sufficient for messageKey=X-GM-MSGID mode. If(msgBHdr.msgDBHdr.messegeKeyMode==X-GM-MSGID-mode), actual_msgDBHdr = Gmail/All Mail[messageKey(==X-GM-MSGID)] ; If(!actual_msgDBHdr){ actual_msgDBHdr=Gmail/Trash[messageKey]; } If(!actual_msgDBHdr){ actual_msgDBHdr=Gmail/Spam[messageKey]; } If(!actual_msgDBHdr){ aThrow Error; } Else actual_msgDBHdr = msgDBHdr; // Same as current Show message data pointed by actual_msgDBHdr;

Changing this bug to meta bug, per comment #0.

FYi. Bug 800003 was for followup of bug 721316 and enhancements after bug 721316. And bug 697409 already existed for request of "mail data in offline-store file of [Gmail]/All Mail only".

Meta bug 800003 was only for enhanceents after (X-GM-EXT-1 use. Meta bug for followup of bug 721316 was bug 798145. Sorry for my confusion.

(In reply to WADA from comment #9) > To Atul Jangra [:atuljangra] (owner of 721316): I am really sorry for replying so late. I don't know how I missed this. :( > Is there good way to improve? On the top of my head, I am not able to find any better way to improve. I'll look at it once again, as soon as I get some free time. I was also planning to resolve some of the bugs related to Bug 721316, but then again, I'll do it as soon as I am free. I am really sorry for the delay. Thanks :-)

If "X-GM-LABELS contains xxx" will be supported by View(Search), and Gmail will resolve problem of bug 1128947, and bug 1123617 will be fixed, following is possible. 1. Server access for [Gmail]/All Mail, [Gmail]/Trash, [Gmail]/Spam only. Upon folder open : uid fetch 1:* Flags X-GM-MSGID X-GM-TRDID X-GM-LABELS Upon new mail check : uid fetch 1:* Flags X-GM-MSGID X-GM-TRDID X-GM-LABELS (CHANGEDSINCE modseq) "Local copy of mail in Offline-Store" is for these 3 folders only => No duplicaate copy in Offline-store. 2. Because all Mbox except above 3 is merely Gmail Label in Gmail, and because "mapping between X-GM-LABEL and MboxName in Gmail IMAP" is known, "MboxName in Gmail IMAP" can be pretty easily known by X-GM-LABEL. 3. If "Offline-Store file for [Gmail]/All Mail, [Gmail]/Trash, [Gmail]/Spam" is similar one to Inbox of POP3 account/Inbox of Local Folders, problem like "Offline-store file content was lost!" won't occur. If user want "sync'ed staus of Mbox with server", it can be done merely by using X-GM-MSGID and X-GM-LABELS. If user want "message filter like POP3", user can create local "Inbox" by filter of "if X-GM-LABELS contains \Inbox, copy to Inbox" :-) 4. Because access to [Gmail]/All Mail, [Gmail]/Trash, [Gmail]/Spam" only, and because fast Gmail IMAP server, Following is possible. (with CONDSTORE only. without QRESYNC) After "Select Mbox" for re-sync : uid fetch 1:* Flags X-GM-MSGID X-GM-THRID X-GM-LABELS Upon new mail check by Biff : uid fetch 1:* Flags X-GM-MSGID X-GM-THRID X-GM-LABELS (CHANGEDSiNCE modseq) When new mail is detected via IDLE : "id fetch nn Flags X-GM-MSGID X-GM-THRID X-GM-LABELS After it, if new UID is detectied : "uid fetch nn BODY.PEEk[]" immediately, instead of "header only fetch + body download by auto-sync". => user can freely use Gmail IMAP as if POP3 which user pretty deeply loves :-)

Sorry, wrong link to bug. I wanted to say: If Gmail will resolve problem of bug 1129870, ...

FYI. Because we already have nsImapMailFolder::GetOfflineMsgFolder, and because Gmail IMAP already supports CONDSTORE, and because Tb already has CONDSTORE support. "Reducing duplicated offline-store data" is possible by following. (1) Keep X-GM-LABELS sync'ed When "fetch Flags" is issued, always use "Flags X-GM-MSGID X-GM-THRID X-GM-LABELS" if Gmail IMAP Until QRESYNC support, use full-resync always : after Select Mbox, uid fetch 1:* Flags X-GM-MSGID X-GM-THRID X-GM-LABELS New mail check : uid fetch 1:* Flags X-GM-MSGID X-GM-THRID X-GM-LABELS (CHANGEDSINCE known_modseq) (2) Fetch from [Gmail]/All Mail only, at many places as possible. nsImapMailFolder::GetOfflineMsgFolder always returns \AllMail folder([Gmail]/All Mail) if Gmail IMAP. If auto-sync of \AllMail folder is possible(auto-sync is enabled and at least one folder is Offline-use=on), force Offline-use=On of \AllMail folder. Force "new mail check by Biff" for \Allmail folder. Force small IDLE timeout for \AllMail folder to avoid problem due to "connection loss while IDLE". After header fetch of new mail at \AllMail folder, "uid fetch m:n BODY.PEEK[]" immediately, before filter invocation. Note: "uid fetch m:n BODY.PEEK[]" is currently issued by filter when Body filter is requested. So, "always uid fetch m:n BODY.PEEK[] like POP3" has no problem. (3) Gloda respects nsImapMailFolder::GetOfflineMsgFolder if Gmail IMAP, Don't blindly look offline flag of mail. Always ask to nsImapMailFolder::GetOfflineMsgFolder for location of offline-store dara if Gmail IMAP. After it, as nsImapMailFolder::GetOfflineMsgFolder recommends \AllMail folder, \AllMail folder is accessed. Outstanding problem. [Gmail]/All Mail can be big folder(Max 10 GB if free Gmail account). So Compact produces performance issue if BerkleyStore and huge [Gmail]/All Mail. So, msgstore/maildir is needed, but msgsore/maildir is still under construction...

(correction of comment #3) > different Gmail Label==absolutely Different IMAP folder, (snip) It was not accurate. true : different Gmail Label => absolutely Different IMAP folder false : absolutely Different IMAP folder => different Gmail Labell Gmail IMAP returned X-GM-LABEL=\AllMail for both [Gmail]/All Mail and top level \AllMail folder in Tb(==user defined Gmail Label named \AllMail). See bug 1129870, please.