I have the following two problems with my Courier IMAP Server. The issues are reproducable with Win NT and Linux. Tested with Mozilla Release 1.0 and Mozilla 1.1a+ Gecko/20020626. The problems do not occur with Sylpheed on Linux and Pegasus Mail on NT: 1. With nested subfolders not all levels are shown, only two. E.g. in the following example subfolder 3-1 and 3-2 do not appear: inbox | -- subfolder 1 | -- subfolder 2 | -- subfolder 3-1 | -- subfolder 3-2 2. When I use the IMAP server directory "INBOX." the folders are no longer subfolders of the inbox, which is fine, but the content is not visible. Error message: "The current command did not succeed. The mail server responded: Invalid Mailbox." When I leave the server directory empty the subfolders are shown below the inbox but the content is visible. Issue 1 happens in both cases.

The Courier IMAP server is running on FreeBSD 4.5 and compiled with --enable-workarounds-for-imap-client-bugs

With Gecko/20020809 the issue with the missing subfolders seems to have improved a little bit: The 2. level of subfolders still do not appear automatically but when I manually add .sbd subdirectories for the missing subfolders to the folder tree in the cache the subfolders then also appear in the folder tree of the mail window.

The subfolders are gone again after closing and reopening Mozilla and this time even creating the subfolders in the cache does not work at all. :-(

Confirm issue 2. Issue 1 doesn't exist, even the message in the nested subfolder is shown. (comment #0) Please verify. Marking new and taking. Add mscott to CC list.

Henry, > Issue 1 doesn't exist Well, for me it does. I am attaching two sreenshots, one accessing the IMAP server with Mozilla and one with Pegasus Mail. Compare the subfolders of folder 'Mailinglisten'. 'cvs', 'FreeBSD' and 'OpenOffice' contain subfolders which do not appear with Mozilla. Something else: I just noticed that creation of folders and subfolders does not work as well. Nothing happens, not even an error message. Is this a known issue? Manfred

Manfred, I'll take more look at this bug. Thx for the feedback. Henry

Manfred, several things. First of all, you should update to the latest trunk build if you want to have the latest Courier fixes. Second, are you subscribed to all those sub-folders? I'm not sure if Pegasus shows you all folders or just subscribed folders. If you get the latest trunk mozilla build, and bring up the imap subscribe UI, you should be able to see if you're subscribed to those sub-folders or not (you need the latest trunk build for that subscribe UI to work, unfortunately). Or, you could go into the advanced imap server settings and uncheck the box that says "show only subscribed folders". If those things don't clear up the problems, an IMAP protocol log might help. Here are instructions for generating a log: http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

bienvenu, > First of all, you should update to the latest trunk > build if you want to have the latest Courier fixes. Do I have the latest trunk, when I use the version from http://ftp.mozilla.org/pub/mozilla/nightly/latest/? Every day before testing I install mozilla-win32-installer-sea.exe from above location. It this is not the right version how do I get the latest trunk? Do I have to compile it myself? > Second, are you subscribed to all those sub-folders? I do not use the subcribe feature currently. > If you get the latest trunk mozilla build, and bring up the > imap subscribe UI, you should be able to see if you're subscribed to those > sub-folders or not (you need the latest trunk build for that subscribe UI to > work, unfortunately). It is not working here: The folder tree is mostly invisible. Only some + and lines at the left can be seen. It does not change anything if I resize tbe dialogue to make it bigger. > Or, you could go into the advanced imap server settings > and uncheck the box that says "show only subscribed folders". Since I want to see all folders I have it unchecked all the time. > If those things don't clear up the problems, an IMAP protocol log might help. Ok, I will switch on the logging. Manfred

The attached log should contain the refresh of the folder tree because I deleted the content of C:\Program Files\mozilla.org\Mozilla\Profiles\Manfred\8c5vhg16.slt\ImapMail\localhost before starting Mozilla and opening the mail window. The updated mail folder tree display still showed no subfolders for Mailinglisten.cvs, Mailinglisten.FreeBSD and Mailinglisten.OpenOffice (The existing subfolders can be seen on attachment 95088 [details].

Manfred Usselmann, I can't read your log. I can't recongnize the format. Would you please supply another one using text/plain format? Thx. If the issue you described does exist, I think we may file another bug. Because issue 1 and issue 2 (comment 0) are not the same and don't have much relationship. Henry

Henry, the attachment is a zip file.Sorry, I thought adding an attachment would preserve the file name. Maybe I should have used another mime type (which?). But I just tested downloading the attachment as log.zip and it worked fine. Could you please try again? I agree that it would have been better if I would have created two separate issues. If you would want to split feel free to do so. Or shall I do it? But isn't issue 2 (comment 0) already covered by other issues any way? Thanks, Manfred

Manfred, I've changed the mime type of your attachment. :-) I'll take care to file a bug for issue 2 if there is not one. Henry

Manfred, I've confirmed your another issue and filed bug 162644. This issue exists not only on courier imap. Change the summary of this bug. Henry

the problem is, from what I can tell in the log, that we're not listing sub-folders when you expand a parent folder in the folder pane. That means that the you more or less need to use imap subscription if you have nested sub-folders. I suspect this has been broken for a while, if not forever in mozilla. The fix would be to make sure we list a folder's children and its children's children when you expand a folder in the folder pane, If and only if we're not using IMAP subscription.

this bug is not related to 160644 - it's just a general problem with the "don't use subscription" feature.

David, For the nested sub-folders showing issue, I filed another bug 162644. It's not a namespace and imap server directory related bug. I also agree with what you said about the reason. I think we can also simplify the process by list all the nested sub-folders once. I mean we can use 'list "" "*"' instead of 'list "" "%" + list "" "%.%"'. Then we need not care about the issue later. I gave out a patch for this issue in bug 162644. Would you please take a look at that and if possible, r=/sr=? Thx. This bug (bug 154778) is related to imap server directory. Because after setting the imap server directory 'INBOX.', you will not get the namespace for the normal folders parallelly with 'INBOX', thus you will not add the imap server directory for converting conanical folder to server side folder. This is the reason for this bug. I'll give out a patch soon. P.S. Would you please sr= bug 161085, bug 137489 and bug 147995 which has passed r=navin? Thx. They are all related to namespace and imap server directory. Regards. Henry

Henry, it's not a namespace issue or a server directory issue. The issue is that we're not listing sub-folders more than two levels deep for any kinds of folders. Listing * is not a good solution - that can be very slow.

Yes, the sub-folders issue is not a namespace related issue. See bug 162644.

This bug 154778 is. See the second part of Comment #17.

Henry, How can we get the namespace for the paralled folders of 'INBOX' when imap server directory is set 'INBOX' (or 'INBOX.a') or etc. ? I only add 'INBOX.' or etc.(which is called onlineDir) when nsImapProtocol::CreateEscapedMailboxName returned . And thus get a little success .

This new method of nsImapProtocol.cpp can let us access those folders parolled with INBOX when IMAP server directory is set 'INBOX'. but mabey it is not a best resolution . I'm working on a new and better one.

Comment on attachment 104620 [details] [diff] [review] Change to nsImapProtocol::CreateEscapedMailboxName Index: nsImapProtocol.cpp =================================================================== RCS file: /cvsroot/mozilla/mailnews/imap/src/nsImapProtocol.cpp,v retrieving revision 1.439 diff -u -r1.439 nsImapProtocol.cpp --- nsImapProtocol.cpp 28 Aug 2002 06:29:15 -0000 1.439 +++ nsImapProtocol.cpp 11 Nov 2002 08:55:40 -0000 @@ -2540,7 +2540,55 @@ // escape any backslashes or quotes. Backslashes are used a lot with our NT server char *nsImapProtocol::CreateEscapedMailboxName(const char *rawName) { - nsCString escapedName(rawName); + //get onlineDir here + nsString aString; + nsCAutoString onlineDir; + m_hostSessionList->GetOnlineDirForHost(GetImapServerKey(), aString); + onlineDir = (char *)(!aString.IsEmpty() ? ToNewCString(aString) : nsnull); + char delimiterToUse, *onlineDelimiter; + //m_runningUrl->GetOnlineSubDirSeparator(&onlineDelimiter); + if (m_imapMailFolderSink) + m_imapMailFolderSink->GetOnlineDelimiter(&onlineDelimiter); + delimiterToUse=*onlineDelimiter; + //delimiterToUse='.';//??? + //get onlineDir ok! + //create the correct string here! + nsCString escapedName("");//ok? + + int len = onlineDir.Length(); + if(PL_strncmp(onlineDir.get(),rawName,len)) + { + //we should get namespaces and compare rawName with namespace prefix + if(m_hostSessionList) + { + PRUint32 count=0; + m_hostSessionList->GetNumberOfNamespacesForHost(GetImapServerKey(),count); + if(count) + { + for(PRUint32 i=0;i<count;i++) + { + nsIMAPNamespace * ns=nsnull; + m_hostSessionList->GetNamespaceNumberForHost(GetImapServerKey(),i,ns); + if(ns) + { + const char* prefix=ns->GetPrefix(); + //compare + if(PL_strncmp("INBOX",prefix,5)) + //if prefix != 'INBOX' + goto done; + } + }//end of for + }//end of if(count) + }//end of if(m_hostSessionList) + + + onlineDir.ReplaceChar('/',delimiterToUse); + if(onlineDir.Last()!=delimiterToUse) + onlineDir+=delimiterToUse; + escapedName.Append(onlineDir.get()); + } +done: + escapedName.Append(rawName); for (PRInt32 strIndex = 0; *rawName; strIndex++) {

1.The attachment 104620 [details] [diff] [review] is not good.It only works on some very special conditions. 2.in Courire server, we will create a wrong dommand when we select a folder. (If a folder 'INBOX.a', and IMAP server directory is set 'INBOX', in Courier server , mozilla will send 'select a' instead of 'select INBOX.a' when we select it.) 3.There are these attributes included in nsImapMailFolder. PRUnichar m_hierarchyDelimiter; PRInt32 m_boxFlags; nsCString m_onlineFolderName; nsFileSpec *m_pathName; nsCString m_ownerUserName; // username of the "other user," as in // "Other Users' Mailboxes" nsCString m_adminUrl; // url to run to set admin privileges for this folder nsIMAPNamespace *m_namespace; // Opaque pointer to the IMAP namespace for this folder // Use libnet accessors for various namespace functionality For this bug, mozilla will get wrong onlineFolderName, so we can't send correct command to server.And mozilla will create folder object and get hostname and other info to create a urlSpec whenever needed. And never save it. 4.Can we add an attribute m_urlSpec in nsImapMailFolder to save the correct urlSpec which will include hostname ,...,onlinePathName,namespacePrefix all these informations. 5.I'm not sure if this can work. And I know the question is that we can't save the object for later use in mozilla. That means we can get correct urlSpec in the beginning we lsub all folders in server, but we can't keep urlSpec for later use.But if we can't get correct onlineFolderName , how will this bug be fixed?:) That's all.:-)

Comment on attachment 104620 [details] [diff] [review] Change to nsImapProtocol::CreateEscapedMailboxName 1. use diff -u instead of the code itself 2. nsImapProtocol::CreateEscapedMailboxName is not a good place. rawName should be right before this funciton is called. You may refer to the code for Cyrus server handling. 3. don't add any new variables, it should be resolved without any new footprint. Thx, Philip. Keep on going.

Comment on attachment 104620 [details] [diff] [review] Change to nsImapProtocol::CreateEscapedMailboxName 1. use diff -u instead of the code itself 2. nsImapProtocol::CreateEscapedMailboxName is not a good place. rawName should be right before this funciton is called. You may refer to the code for Cyrus server handling. 3. don't add any new variables, it should be resolved without any new footprint. Thx, Philip. Keep on going.

Comment on attachment 104620 [details] [diff] [review] Change to nsImapProtocol::CreateEscapedMailboxName Sorry for the spam. Not sure why there is two same flags.

This bug will also has a perfomance like this: 1.Clear the IMAP server directory setting. 2.create a folder Inbox.Inbox 3.we will see Inbox.Inbox.Inbox folder hierachy now.Save a message AAA to Inbox.Inbox.Inbox 4.now set the IMAP server directory ot 'INBOX' 5.refresh the folder pane 6.we will see just Inbox.Inbox folder now 7.select the Inbox.Inbox folder Result: 1.we can select it 2.But the message we get is not AAA,but the message originally exits in Inbox!

> 2.But the message we get is not AAA,but the message originally exits in Inbox! should be : 2.But the message we get here is not AAA. The message we get here will be those messages exists originally in Inbox.That means we can't access the content in Inbox.Inbox.Inbox.

Add comment to attachment (id=106487) and this bug below. 1.This bug cause serious error when user configured the IMAP server directory. 2.On UW server and iPlanet server, we prefer the user not set IMAP directory. When the IMAP directory is set on these 2 kinds of server, dispaly will be wrong( user will can't see those folders parelled with INBOX originally). This is because those Drafts Templates and other many folders we created are parelled with INBOX first. The setting of IMAP directory will make them dispear. 3.On Courier server and Cyrus server ,user can set IMAP server directory in some cases. 4.On Cyrus server, the setting of IMAP server directory to 'INBOX' will not cause error because original code add onlineDir before the foldername we selected (or used in other case). 5.On Courier server, this bug exists because the need-added onlineDir in not add before foldername we selected. 6.attachment (id=106487) resolute this question. It add onlienDir before foldername when necessary. 7.And the test result of this patch is: 7.1 Whatever IMAP server directory is set 'INBOX' or something like 'INBOX.a' it all works. 7.2 If we set PersonalNamespace (or other kinds of namespace) manuelly and cleared the override_namespace flag , it works well. 7.3 If we don't cleare the override_namespace flag , it works well. 8.Shortcoming of this patch: It doesn't relolute the question in comment #28 and comment #29. 9.While I think, the question in comment #28 and comment #29 will not have a good resolute if we don't know the exact hirachy level of a folder.

I also think,mabey we shouldn't use rv = hostSessionList-> GetNamespaceForMailboxForHost(serverKey,onlineMailboxName,ns); at all. For you see, if the prefix of onlineMailboxName is the same with the prefix of a special namespace (whatever 'Inbox.' or 'user.' or 'shared.'), the code will cause error and the result will be we still can't access the content of the folder we selected. And this is also the reason of question in comment #28 and comment #29.

Comment on attachment 106487 [details] [diff] [review] add a patch > When the IMAP directory is set on these 2 kinds of server, dispaly will be > wrong( user will can't see those folders parelled with INBOX originally). We can't think the display is wrong. This is because you want to omit the online part of you directories showed. After you code goes to +addonlinedir: and ns is null, mozilla will crash.

I add 'if(ns)' before 'goto'. This may avoid mozilla's crash when ns is NIL. A question,henry. If the folder hierachy is INBOX.a.b.c, how can we know INBOX is at the level 0, and folder 'a' is at level 1, and 'b' at level 2, 'c' at level 3 (whatever we set 'INBOX' as the IMAP server directory, they are all should be at this level)?

Comment on attachment 106552 [details] [diff] [review] add if(ns) on patch 106487 You shouldn't only get the personal namespace. The work around for this bug is to add a "" namesapce and uncheck 'Allow server to override these namespaces' in the 'Advanced IMAP Server Settings'.

This patch considered other user's namespace and public namespace. We need only the delimiter of certern namespace we get(since there may be different delimiter for different namespace).

Comment on attachment 107465 [details] [diff] [review] new patch for test What do you think if the online directory is set to 'INBOX' instead of 'INBOX.'? Also, no need to + if(ns) + rv =hostSessionList->GetDefaultNamespaceOfTypeForHost(serverKey, + ns->GetType(), + ns);

Hi, henry, Thx you! 1. > What do you think if the online directory is set to 'INBOX' instead of > 'INBOX.'? I think both 'INBOX' and 'INBOX.' will let us find the namespace because of these lines in nsIMAPNamespace::MailboxMatchesNamespace. > if (PL_strstr(boxname, m_prefix) == boxname) > return PL_strlen(m_prefix); > > // If the boxname is part of the prefix > // (Used for matching Personal mailbox with Personal/ namespace, etc.) > if (PL_strstr(m_prefix, boxname) == m_prefix) > return PL_strlen(boxname); 2. >Also, no need to >+ if(ns) >+ rv >=hostSessionList->GetDefaultNamespaceOfTypeForHost(serverKey, >+ ns->GetType(), >+ ns); I think we may need them because what the 'ns' we get using nsIMAPNamespaceList:: GetNamespaceForMailbox is the last one who matchs the mailbox name in namespace list. While the 'ns' we get using nsIMAPNamespaceList::GetDefaultNamespaceOfType is the first one who matchs the mailbox name in namespace list. And I think the first to be used as default is better. philip

In our firstbase server , namespace is : a namespace * NAMESPACE (("" "/")) (("Shared Folders/User/" "/")) NIL a OK Completed The patch is ok because of the ("" "/") namespace. If the namespaces is like * NAMESPACE (("INBOX/" "/")) (("Shared Folders/User/" "/")) NIL and the onlineDir is set 'Shared Folders' or 'Shared Folders/Users',the patch is also ok.

This patch disable user's configure to IMAP server directory 'INBOX' or 'INBOX.' etc. when server has a personal namespace 'INBOX.'. Thus avoid many problems on IMAP server directory setting. And I think we also need attachment 107465 [details] [diff] [review] because we still keeps IMAP server setting now. And user still can configure personal namespace and uncheck override namespace, then configure IMAP server directory again. Thx for David's help much. :-) Pls test the patch. We may need some more talk about the architechture.

Instead of comparing both the personal namespace and the online server directory to inbox, I suggest comparing them to each other, and not special casing INBOX like this. And instead of keeping attachment 107465 [details] [diff] [review], I think we should disable the IMAP Advanced Server Settings UI to prevent the user from doing what you describe. Finally, one user who had this problem told me that they set the imap server directory so that sub-folders of the INBOX will appear indented under the INBOX as sub-folders, and that this doesn't happen with the personal namespace approach. So we might want to change that - though I could have sworn I've had people complain the other way too - what do you think?

Hi,David, I will add a new patch to let personal namespace and onlineDir to compare to each other soon. And if there are more than 1 personal namespace on the server, I will make onlineDir to compare to the defaut one(ie.the first one in the namespaceList). If you don't think so, pls let me know. And to attachment 107465 [details] [diff] [review], I agree to delete IMAP server directory and not use the attachment. And at that time, we may rewrite the method AddOnlineDirIfNecessary or delete it. Thirdly, I know that IMAP server directory is working now as the filter of folder names. If we set it, mozilla will create a different folder struct in the client side. The difference of the folder hierarchy in client side and server side gives us much trouble. That's why we delete IMAP server directory. And if we make personal namespace setting work as IMAP server directory, the same question will be still there. So I don't think we need let personal namespace setting to act as IMAP server directory. If you don't think so, pls tell me too. philip

Phillip, I think what you're proposing sounds OK, though I'll have to see the patch to be sure that I'm understanding you correctly. Also, bear in mind, the biggest reason we have trouble when the user sets the imap server directory to INBOX. on a server that returns INBOX. as the personal namespace is that it's simply wrong :-)

Hi,David.Sorry for the delay. :-) This patch has changed as your suggestion. It compare onlineDir with default personal namespace and disabled user's configure to 'INBOX' when server has a personal namespace 'INBOX'. In other conditions, for exam, on Cyrus server when we set IMAP server directory to INBOX and server has a personal namespace "", IMAP server direcotry still works to let INBOX's subfolder listed as parelled with INBOX. Pls see if this patch is ok. Thx. :-)

Add a check if the namespace is null. Pls see it, David. thx.

How about set server directory directly to "INBOX" by ignoring case in nsImapIncomingServer::SetServerDirectory if we really need to fix this bug ?

> How about set server directory directly to "INBOX" by ignoring case in > nsImapIncomingServer::SetServerDirectory if we really need to fix this bug ? Should be: How about set server directory directly to "INBOX" by ignoring case in nsImapIncomingServer::SetServerDirectory if we really need to fix bug 160643? :-)

If anyone needs test account(s) on a Courier IMAP server to test these patches against (as I don't have the facilities here for rolling my own builds), send me an email and I can arrange one for you.

+ PL_strlen(onlineDir.get()))) should be onlineDir.Length() + if ( ns ) + result.AssignWithConversion(PL_strncmp(host->fOnlineDir, + ns->GetPrefix(),PL_strlen(host->fOnlineDir)) ? + host->fOnlineDir: ""); + else + result.AssignWithConversion( host->fOnlineDir ); + } this should just be if (ns && !PL_strncmp(host->fOnlineDir, ns->GetPrefix(), PL_strlen(host->fOnlineDir)) result.AssignWithConversion(""); else result.AssignWithConversion(host->fOnlineDir); or you could even make it one statement, right? // if onlineDir is same as namespace prefix, return empty onlineDir, // otherwise, return onlineDir. + result.AssignWithConversion(ns && !PL_strncmp(host->fOnlineDir, + ns->GetPrefix(),PL_strlen(host->fOnlineDir)) ? "" + : host->fOnlineDir);

Thanks David! I change it.

attachment 114746 [details] [diff] [review] disable user's configure IMAP server directory to 'INBOX' or 'INBOX.' etc when the IMAP server which we connect has a personal namespace 'INBOX.'.

Comment on attachment 114746 [details] [diff] [review] patch disable user's configure to 'INBOX' v4 sr=bienvenu, if you fix the brace indentation to be like the rest of the file e.g., if (a) { ... } instead of if (a) { } thx

I believe this is fixed now. Marking fixed.