Observed with 2001050608 on Windows 2000. Summary: Threads with an expired message will show the unread thread indicator, even when all are read. Steps to Reproduce: 1. Visit a newsgroup which has a thread that contains unread postings, but has an expired message. 2. Expand the thread, and read all of the postings. 3. You can even try Message | Mark | Thread as Read, but this doesn't work, either. Expected Results: When you read all of the messages in a thread, the thread icon shouldn't have a green "unread" indicator on it. Actual Results: We show the message status as being "read" in the columns to the right, but the icon doesn't update to reflect that we've read the whole thread.

nice catch, stephen top of my head guess: I bet when we mark the "expired" message as read, we are unable to find thread it belongs to and so we fail to adjust the unread count on the thread. but I'd have to debug to find out. cc'ing bienvenu.

Or, we're marking it read without going through the db (e.g., just poking the hdr flag), and thus not adjusting the thread unread count.

bienvenu's scenario sounds more likely. accepting.

weird. at first, I was unable to reproduce this. but then (using Mark | Thread | Read?) I was able to get into a bogus state where the unread count on the thread was clearly wrong. stephend, when you see this problem, what's the unread count for the thread? (it's in the "Unread" column, second from the right in the thread pane)

related weird note: when I get into this bad state, my unread count is negative (-1 in my case) and that causes the green icon in the thread column. nsMsgDBView.cpp, line 4287, we ask the thread for the unread count which is -1. but it's an unsigned int, so -1 is really a big int, which is > 0 so we show unread. Investigating...

The unread count for that thread indicates 1, so I'm assuming that would be the expired message.

can you attach a screen shot so I can see if it is what I'm seeing?

ok, I think we're seeing the same thing. start up, find a thread with a read, expired article. click on the green dot to mark that expired article as "unread". the unread count for the thread doesn't change. after the first click, it works fine. but the initial change doesn't work so the unread count on the thread is off by 1. I'll work on that scenario first.

Yeah, sorry, in fact I had played with that (toggling the status icon) but forgot to include that in my original filing. Thanks for clarifying that, Seth.

the first time you mark an expired message as unread, it's not in the read set, so we think it is already unread. we've got code that will bail out if you are trying to mark an unread message as unread (or a read message as read). see nsMsgDatabase::MarkHdrRead() still working on this... I think nsMsgDatabase::MarkHdrRead() is correct. I need to investigate why the expired message isn't in the read set.

weird, looking in my news.rc file, I have something like this: 1-5000,5773,5774 5773 is my thread root, 5774 is my expired thread child. I would have expected to see 5773-5774. perhaps that's at the root of the problem. (the reason it would work on subsequent clicks is I think the unread set code caches the last thing.)

taking, I'm looking at these issues.

I changed the underlying db methods to relax the restriction on not marking msgs read that are already read, in the case that the hdr flags are set differently than the newsrc flags. (the !! stuff is because you need to make sure booleans have been normalized to 1 and 0 before comparing them for equality). I also cleaned up a lot of unused stuff in nsNewsDatabase.cpp. Cc'ing Navin for review and asking Seth for sr. (I'm not really sure this patch fixes this exact problem, but it is good for other stuff too - this problem might be fixed by other changes as well.).

sr=sspitzer

r=naving

Navin and Seth, can I get a couple more reviews on the last patch? It just makes sure we use the newsrc flag setting instead of the db setting, and moves some common code into a function. The common code also makes sure the db flags correspond to the newsrc flag when we open the view.

r=naving

sr=sspitzer

David, do you have anymore cleanup work on this bug? 1.65 bienvenu%netscape.com Aug 2 06:43 more work on syncing news read state between db and newsrc r=naving, sr=sspitzer 79130 I'll take a look at the client and see if my problem is fixed, but should this be marked fixed for me to verify now?

Stephen, you can look into it. I've checked in various fixes to sync up the state but I don't know if will fix this particular case of the threads with expired messages.

This still occurs with the 2002-01-03-03 build on Windows 2000. Here's the behavior remaining: If an expired article exists in a thread, and you read all of the messages in the thread, we still display the unread thread icon. If you run the list-id? url on the newsgroup, and come back to the thread, we still display the unread thread icon. I don't think this is major, especially since it's news, and happens mostly with expired articles.

Still see this with BuildID 2002061018 on Red Hat Linux 7.2 (but for total counts, have not checked thread counts). It appears that the newsrc consideres these messages to be unread, while db does not.

I sure hope this is the active bug for this. I think I have a little insight into this bug. My symptom is that a newgroup will default to some number (say 9). If more messages actually come in, the reported number of unread messages in the sidebar will be 9 plus the number of new messages. I was given a "fix" for the bug, which was deleting the file specific to the newsgroup and restarting Moz. It almost worked. Now, the newsgroup title is just bold normally. When I select it, it now displays a 1 in parenthesis, even though there are no new messages. Also, it seems that it is directly related to the number of posts I've made on that newsgroup. I only get this bug for newsgroups that I've posted on (I read a lot, and post little). I've been reading a lot of posts for this bug, but I've never heard it mentioned that this bug interferes with the "next" button. It won't move past the thread with the false positive in it.

This bug also occurs in build 2002082608 (1.1 final) running on Windows XP.

*** Bug 160963 has been marked as a duplicate of this bug. ***

I've frequently experienced a bug resembling this, but I'm not sure if this is it. I certainly haven't pinpointed the cause to expired or cancelled messages. Sometimes a thread will appear as having unread messages (both the green arrow and the underlining) even though there are none. They even clutter the Threads with Unread and Watched Threads with Unread views. In some cases, expanding the thread and then collapsing it clears the status; in others, the thread is stuck forever in this damaged state. The only way to get rid of it seems to be to either unsubscribe and resubscribe or to delete the .msf file, thereby losing all watched/ignored threads and downloaded message bodies. Indeed, I am experiencing it on a news server that never expires messages - so obviously it isn't expired messages causing the problem, but could still be something to do with cancelled messages. And so waiting for the whole thread to expire and it stops getting in the way isn't an option. Maybe having a sample .msf file that shows the problem would help. I'll try to supply one next time the problem comes up.

I'm experiencing the same problem described by Stewart Gordon, with additional caveat that subscribe/resubscribe does not seem to work every time.

I'm still experiencing this problem with version version 1.5.0.9 (20061220). I synchronize the .rc file across several computers, and then read the news from all these computers. unread messages are correctly highlighted. So if I show only unread messages I actually have only the unread messages. But if I show Threads with Unread, I get also threads where all messages are read, in particular if on that computer the first message of the thread was never opened (it was read on another computer).

I've been seeing this problem for about as long as I've been using Mozilla products, and I still see it in TB 2.0. What appears to be happening is that the .rc file for the affected news server is not being correctly updated. I can correct the problem for a newsgroup that should be all read but shows messages as unread in the folder pane by manually editing the .rc file outside of the application, and changing the line to <newsgroup name> 1-<lastnum> When I go back into teh app, teh counts will be correct. What I haven't been able to isolate is why random lines in .rc files aren't being correctly updated. ______ Dennis

JFY. Accordigng to meta Bug 71728, Bug 24592 & Bug 34406 seem to be oldest ones for download dialog, and Bug 41457 seems to be oldest one for newsgroup count display.

JFY. Bug 108650 seems to be oldest one for different count before open newsgroup from count after open newsgroup.

Adding "incorrect count" in summary to reflect symptom of DUPed Bug 160963 by comment #30 for ease of search. David(bug owner), please correct it, if this bug is only for "unread indicator for already downloaded article which was canceled or expired after download".

Are you sure the problem is in the .rc file? As far as I can see, those files are updated correctly, in fact, if I use "show unread messages" I actually see only the messages that are not read. The problem is when I use "view thread with unread", and I think the problem is due to the fact that thunderbird uses the .msf and .dat files instead of actually inspecting all messages in a thread (e.g., compare the message id against the ids of the .rc file). This happens to me since I read newsgroups from the same server on different machines and I always use the same .rc files (I keep them synchronized), while the .msf and .dat files are different on the different machines. Thus, if I mark all messages of a thread on a machine, copy the .rc file onto another machine and reopen the newsgroup, the same thread is shown even if all of its messages are read. I don't know about the internals of thunderbird on this subject, but I suspect that this is the problem... you may also reproduce this behavior by simply remove the .msf and .dat file of a newsgroup, then reopen thunderbird, reopen that newsgroup, choose "view threads with unread" and see this wrong behavior... This bug makes thunderbird very hardly usable for reading newsgroup, and it's quite astonishing it hasn't been fixed yet... if, at least, someone could provide information about the part of code responsible for dealing with threads with unread one could try to fix the problem... thanks in advance

(In reply to comment #41) > Are you sure the problem is in the .rc file? [...] I am (or at least that's where "part" of the problem lies): messages canceled before having been read cannot be "marked as read" in the rc file other than by hand-editing, and they are counted as "unread" in the newsgroup in the left pane; see bug 294754 comment #3.

(In reply to comment #42) > (In reply to comment #41) > > Are you sure the problem is in the .rc file? > [...] > > I am (or at least that's where "part" of the problem lies): messages canceled > before having been read cannot be "marked as read" in the rc file other than by > hand-editing, and they are counted as "unread" in the newsgroup in the left > pane; see bug 294754 comment #3. I don't know whether it's where the actual problem is, but it's certainly the visible evidence. Newsgroups are displayed in the folder pane in the order in which they appear in the .rc file, and read/unread counts are stored in the line for each individual group. (I've re-arranged the order in which broups for a server are listed by editing the .rc file for that server -- it would be nice if there were a GUI interface to do that...) Manually correcting the appropriate lines in the .rc fixes the innacurate read/unread display in the folder pane. I have no idea where in the TB code that functionality resides, or whether it's in the code that actually updates the rc file, or the code that keeps the info used to do so. Unlike Tony, I haven't tried to cancel messages. My problems occur simply in reading. The read counts are not always correctly updated. I wouldn't go so far as to say it makes TB unusable for reading news -- it's more of an annoyance -- but since I only use TB as a newsreader, and handle email elsewhere*, it's a *big* annoyance. * I use GMail as my primary email account, and prefer the web interface. I don't get email as POP mail, though it's nice that TB 2 adds GMail POP support out of the box. (It would be nicer still if it did not default to trying to download mail as POP as soon as you set it up. You might not want to do it right then, or at all. The setup process for GMail should mention that is is setting things up for POP delivery, and let you specify when it should be done.) ______ Dennis

(In reply to comment #43) [...] > Unlike Tony, I haven't tried to cancel messages. [...] I haven't either, but I've had messages canceled on me (by moderators or whomever) before I had come around to reading them, causing the problem.

Sorry if I insist on this: I think the .rc file is updated correctly, it is the view "threads with unread" that is not correct. Here's how to reproduce the problem: 1. subscribe to a new newsgroup (e.g., alt.test) 2. open it and download the headers (even a part of it, if they're too many) 3. mark all as read 4. close thunderbird 5. remove the .dat and the .msf files 6. re-open the newsgroup 7. select "thread with unread" and all the threads will be shown even if all the messages are read As I said, this happens to me since I read newsgroups from the same server on different machines and I always use the same .rc files (I keep them synchronized), while the .msf and .dat files are different on the different machines. Thus, if I mark all messages of a thread on a machine, copy the .rc file onto another machine and reopen the newsgroup, the same thread is shown even if all of its messages are read. The steps above is only to simulate this situation...

(In reply to comment #46) > Sorry if I insist on this: I think the .rc file is updated correctly, it is the > view "threads with unread" that is not correct. Sorry if I insist on this: I never use the "Threads with unread" view, I only use "View All", yet at times there are newsgroups which appear bolded and with a number in the left pane; then when I select that NG, there is no unread msg after all, except as soon as I move away the bold & count reappear. The only way I've found to make the problem disappear is by hand-editing the newsrc as I originally posted at bug 294754 comment #3. The way I see it, the "deleted" or "expired" message reappears as unread when the counts are refreshed and cannot be marked permanently as read from within Thunderbird.

(In reply to comment #46) > Sorry if I insist on this: I think the .rc file is updated correctly, it is > the view "threads with unread" that is not correct. Have you actually *looked* at the .rc file when this happens? The .rc file is *where* the information on what is and is not read displayed in the folder pane is stored. If you bring up the .rc file in an editor on a group where you've marked everything as read and posts still show as unread, you'll see the problem. Here's a sample .rc file from my system, for newsgroups on the Baen server: baen.1632Tech: 1-155504,155512,155516-155518,155526,155534,155543 baen.administrivia: 1-15915 baen.bar: 1-61263,61268-61270,61275,61277 baen.books: 1-17530 baen.buships: 1-28405,28420,28422,28423,28428,28430,28432-28434,28437 baen.classicsf: 1-1293 baen.EBookReader: 1-1836 baen.faq: 1-91 baen.FutureTech: 1-14769 baen.honorverse: 1-39405 baen.space: 1-4431 This: "baen.space: 1-4431" indicates all messages are read. This: "baen.bar: 1-61263,61268-61270,61275,61277" indicates that messages 1-61263 are read, 61264-61267 are *not* read, etc. Look at the .rc file for a group which shows an unread message count in the folder pane but all messages are in fact read, and you'll see that the line in the .rc file for that group is wrong. When I see that problem, I can correct it by editing the .rc file to include the correct range of read messages. When I restart TB, the problem is gone. What I *can't* reproduce is *why* a particular entry in the .rc file isn't properly updated. It happens randomly, occurs on any of the six news servers I have configured, and doesn't affect all groups on the server. ______ Dennis

(In reply to comment #48) [...] > This: "baen.bar: 1-61263,61268-61270,61275,61277" indicates that messages > 1-61263 are read, 61264-61267 are *not* read, etc. [...] They aren't read, and, in the case concerned by this bug, cannot be read because they don't exist anymore. I think there are in theory two possible fixes for this bug: a) Leave the rc as it is, but don't count messages that don't exist on the server anymore when computing the number of unreads for a group; or b) when it is found that a message doesn't exist anymore, mark it as read in the rc file so the mailer will remember that that message isn't "unread" (in the sense of being available for reading). I think (b) would be easier to implement, since (a) wouldn't leave a memory of which "holes" in the list correspond to messages that cannot be read anymore.

(In reply to comment #49) > (In reply to comment #48) > [...] > > This: "baen.bar: 1-61263,61268-61270,61275,61277" indicates that messages > > 1-61263 are read, 61264-61267 are *not* read, etc. > [...] > > They aren't read, and, in the case concerned by this bug, cannot be read > because they don't exist anymore. I think there are in theory two possible > fixes for this bug: > > a) Leave the rc as it is, but don't count messages that don't exist on the > server anymore when computing the number of unreads for a group; or > b) when it is found that a message doesn't exist anymore, mark it as read in > the rc file so the mailer will remember that that message isn't "unread" (in > the sense of being available for reading). > > I think (b) would be easier to implement, since (a) wouldn't leave a memory of > which "holes" in the list correspond to messages that cannot be read anymore. I agree that (b) would be easier to implement. But the basic issue is that the .rc file is wrong. For the purposes of this discussion, it doesn't matter whether the messages exist and *are* all read, or no longer exist on the server and can't be read. The .rc file is showing unread messages when it should not. ______ Dennis

(In reply to comment #48) > (In reply to comment #46) > > Sorry if I insist on this: I think the .rc file is updated correctly, it is > > the view "threads with unread" that is not correct. > > Have you actually *looked* at the .rc file when this happens? > > The .rc file is *where* the information on what is and is not read displayed in > the folder pane is stored. > the .rc file is good, in fact if I switch to the view "show only unread messages" I actually see no message (since all are read): the problem is only with the view "show threads with unread"... probably I'm posting in the wrong bug?

See comment #7 in issue 365819.

Please do proposal (b) in comment 49! I think this would fix SO MANY problems. Maybe even bug 400526.

(In reply to comment #49) > a) Leave the rc as it is, but don't count messages that don't exist on the > server anymore when computing the number of unreads for a group; or > b) when it is found that a message doesn't exist anymore, mark it as read in > the rc file so the mailer will remember that that message isn't "unread" (in > the sense of being available for reading). Recalling a bit from memory, the newsrc file is written by dumping the key set. I can see the value of remembering "holes" on a bigger scale, since it would be nice to know that we haven't tried looking at a block of articles. But on the other hand, if we download articles 1-1000 and don't see article 334, it's safe to assume that we won't ever see 334. Questions to ponder on: * What criteria should we use to determine which missing messages should be glossed over in the key set? ** (Tentative response: XOVER-time. But what about interrupted downloads?) * How should Mark All As Read work? ** (Tentative response: reset the newsrc to <oldest>-<latest>. But how does this interact with Get Next N Messages?) * What if we do find 334 later on? ** (Tentative response: Mark it as unread and reset read set information) Reassigning to myself (and resetting QA). Hope bienvenu doesn't mind :-).

Hit commit too early (and need to accept bug anyways)... In any case, it looks like b) is the way I'm going to go, although it's a b-with-comments.

The only portion of the summary I can verify is the incorrect message count resulting from message cancels. This I have seen on news.annexcafe.com, a server that honors user news cancel requests. The other case involving that server was cancels of spam, which have been curtailed by the server going to all users requiring authentication. Then My low traffic group was effected often. Seemed like message numbers were getting recycled. Possibly involved some server reindex. Them a cleanup of the *.rc file was not enough and clearing the effected *,msf was needed.