Closed Bug 56007 Opened 24 years ago Closed 24 years ago

certain unfiltered message in inx show wrong message and gives "

Categories

(MailNews Core :: Filters, defect, P1)

x86
Linux
defect

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: philipp.stucke, Assigned: Bienvenu)

References

Details

(Whiteboard: [rtm++]5587155871)

Attachments

(1 file)

Mozilla M18 Mozilla/5.0 (X11; U; Linux 2.2.16 i686; en-US; m18) Gecko/20001010 Ok, this is a bit complicated, but it makes mozilla news unusable for me since important messages get lost. Scenario: I have 2 POP3 Accounts. Several accounts are forwarded to the 2 POP-Accounts. I grab all E-Mail and (of course) filter them to several Folders. (E.G. Maillinglists, etc.) The only mail that is unfilitered comes to my Inbox (so all unfiltered Mail is private mail). When I see such a Mail in the Inbox folder, the Subject and the Sender is there correct. But when I scroll with the cursors (no matter if I click on it by mouse directly), and I reach the mail, the header is correct, but it shows me some mail before or after this mail. The mail is completely lost then, I never ever can see it again. (so this makes mozilla hardly usable for me). If i search in the "plain .mozilla" directory the Mail file, the message is not there. Sometimes a window from mozilla pop's up saying "bin: unknown error". To sum it up: I believe this is due to my setup in some way: I grab mail with (example) a@pop3.de, BUT, since most E-Mail are forwarded to this, "originally" the mail is for b@bla.com. I feel all this mails get lost, which are in the header to b@bla.com BUT which I pickup at a@pop3.de. Strange thing: ALL (!!) message I filter are going to the correct folder, and there this "mail loss" never ever happened. JUST in the "Inbox" folder, some Mail gets "lost" somehow. Its MAYBE (!) something with #46888 or/and #55432 Reproducible: Always Steps to Reproduce: See Description: I get a Mail, see it in Inbox, click on it, but get either bin:unknown error (pop-up) or see some other Mail (again, correct header, but wrong body from some mail above). I get this error every time, but I dont know how I should tell you to reproduce it since it "just happens anytime" at my computer. Actual Results: Error, or wrong Mail. Expected Results: Showing me the correct Mail. I'm sorry that I can't be more concrete especially on "Reproduce" Part, but somehow it must be something with a) either filters or b) different Aliases on one POP Account. Feel free to contact me, I want to help more! Its a really bad problem since mail gets lost.
this is another backend mail filter message dataloss bug. we really need to figure out what's going on.
Assignee: alecf → bienvenu
nominating for rtm, but I really don't know how to reproduce it.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Keywords: rtm
QA Contact: esther → laurel
I don't know if this will help, but the "unknown error" has to date been seen in instances where Get Msgs process was interrupted or failed. Just today I was working to verify a bug about hitting disk full conditions which produced the Unknown Error for POP on linux. In the disk full situation, the message will eventually disappear... so maybe a write to disk problem was encountered which interrupted the message retrieval? I don't know ...
the unknown error occurs when we're trying to read a message that is not as big as we think it is, and we hit the end of the file before we read all of the message. This can occur for two reasons: we have the wrong message size, or we have the wrong "start of message", meaning we have the wrong offset into the folder for the start of the message. But I still don't know what causes this.
accepting. So you're sure that the message you can't read is not actually in the Inbox file at all? You tried grepping through your Inbox for that message? If you shutdown mozilla, move away your Inbox.msf file (put it in a temp directory or something), and then reopen your inbox, is the header for the message still there?
Status: NEW → ASSIGNED
Priority: P3 → P1
Ok, I did some more testings on this and HOPEFULLY can give you the way how to reproduce this very ugly bug :). My first though was correct, it has to do with the aliases and filtering. a) Setup an Alias (a@b.de) which is forwareded to your POP3 (pop@c.de) b) set filter rules for a@b.de (--> in my situation: Maillist) c) if no rule matches, it stays in Inbox (--> Private Mail) d) E-Mail a@b.de with NO appropriate filer (aka Private) so it stays in Inbox e) Grab your POP3 Mail from pop@c.de f) You'll see the Mail in Inbox, to a@b.de g) You click it --> BINGO --> Unknown error. Tested several times right now. If I mail pop@c.de directly with NO filter rule (aka Private) I can READ it, it acts normally!! I STILL need to test what will happen to my other POP3 account (only Priv Mail there so no filter) when I create some forwards to it (so we can see if it is a filter problem, or some other problem). And, yes I am abled to use grep correctly :) The message is LOST, it is not in Inbox, and when I move Inbox.msf and restart, YES, the header is still there! Ok, hope this helped, thanks for the fast replies, let's hit the target now why this is buggy :)
Ah, I forgot, of course, my HD isnt full, and no, I didnt interrupted the GET message process at all.
Ah, I forgot, of course, my HD isnt full, and no, I didnt interrupted the GET message process at all, neither it went wrong. The unknown error appears (if I understood correctly) because it tries to read a Mail that doesnt exist in the Inbox file. Ok. Thats easy. So the mail gets lost before, I guess in the filtering process. But why is it lost in space?
Now I'm very confused - if you delete the .msf file, and restart, we will reparse the mailbox, and only show you headers for messages that are in the mailbox. So you must have at least some of the message in your mailbox if you still see headers after deleting the mailbox. The unknown error is not necessarily because the msg is not in the mailbox; it can be that the database (the .msf file) has the wrong size or offset for the message. When you moved away the .msf file, we will reparse the mailbox and almost certainly have the correct offset and size. Do you still see the unknown error after moving away the .msf file? If so, this isn't a problem with filtering per se.
Can you attach the filter you described (it's in a file called rules.dat) and also a sample forwarded message so I can see what they look like? thanks!
Ok, to #1 of your questions, I'm sorry, *MY FAULT*. If I delete Inbox.msf, the messages won't show up anylonger in Inbox (thats logic: They definitly dont exist in Inbox anymore, so its logical if Inbox.msf gets rebuild, it wont show up in the rebuild Inbox), all as you said. *SORRY* I can 100% reproduce the error now (sending to the alias Adress of the POP3 account.) Here is the DAT file with the rules. version="8" logging="no" name="P-Privat" enabled="yes" type="1" action="Move to folder" actionValue="mailbox://me@mypop3account.de/Inbox" condition="OR (to,contains,me@someotheradress.com)" name="SuSE" enabled="yes" type="1" action="Move to folder" actionValue="mailbox://nobody@Local%20Folders/SuSE" condition="OR (to,contains,suse-security@suse.com)" name="Bugtraq" enabled="yes" type="1" action="Move to folder" actionValue="mailbox://nobody@Local%20Folders/Bugtraq" condition="OR (to,contains,bugtraq@securityfocus.com)" name="Vuln-Dev" enabled="yes" type="1" action="Move to folder" actionValue="mailbox://nobody@Local%20Folders/Vuln-Dev" condition="OR (to,contains,vuln-dev@securityfocus.com)" name="Pen-Test" enabled="yes" type="1" action="Move to folder" actionValue="mailbox://nobody@Local%20Folders/Pen-Test" condition="OR (to,contains,PEN-TEST@SECURITYFOCUS.COM)" name="Incidents" enabled="yes" type="1" action="Move to folder" actionValue="mailbox://nobody@Local%20Folders/Incidents" condition="OR (to,contains,INCIDENTS@SECURITYFOCUS.COM)" Hope it helps.
I have to add this while playing: If i delete the first filter rule, the error is NOT showing up anylonger ?!?. (I mean that's fine, but I still dont see why the original error before occured. Maybe you are knowing more ?!?)
Is the rule that moves to this folder : "mailbox://me@mypop3account.de/Inbox" moving to the inbox of the account you're getting new mail for? In other words, is it a NOOP, just to stop other filters from firing? Or is it the inbox of another account/profile? Also, could you attach at least the headers of a message that gets mis-filtered? thanks!
I will try setting up a "move to inbox" filter for pop3 and see what happens. Thanks for all your help!
Hehe, damn, I was just making the post ready for your other question, and then I got "collusion: Someone else is editing and all went away". OK, we agree (I guess *g*): It'a NOOP filter which makes Mozilla worry. If you still need the header, here it is: (**** too lazy to annonymize ****) Return-Path: <wilkins@sekure.net> Received: by Mail.ZEDAT.FU-Berlin.DE (Smail3.2.0.98) from webirc.de (192.41.57.212) with smtp id <m13jPbY-00O908C>; Wed, 11Oct 2000 19:23:48 +0200 (MEST) Received: from birdie.sekure.net (wilkins@sekure.net [193.15.98.64]) by webirc.de (8.8.5) id TAA18095; Wed, 11 Oct 2000 19:23:35 +0200 (CEST) X-Authentication-Warning: webirc.de: Host wilkins@sekure.net [193.15.98.64] claimed to be birdie.sekure.net Received: from localhost (wilkins@localhost) by birdie.sekure.net (8.10.1/8.9.3) with ESMTP id e9BHSMD20161 for <wilkins@pimmel.com>; Wed, 11 Oct 2000 19:28:22 +0200 (CEST) Datum: Wed, 11 Oct 2000 19:28:21 +0200 (CEST) Von: wilkins <wilkins@sekure.net> An: wilkins@pimmel.com Betreff: bleh Nachrichten-ID: <Pine.BSO.4.21.0010111928080.15072-100000@birdie.sekure.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Mozilla-Status: 8001 X-Mozilla-Status2: 00000000 X-UIDL: IGH!!J)F!!$_=!![*O"!
Attached patch proposed fix (deleted) — Splinter Review
we need to fix this; there's data loss, can I get an rtm+ so we can get this through the approval process?
David, the prevalence of this bug being hit depends on the number of people who would need to create such a nop rule in their filters right? We think that could be a significant percentage of filter users because it's sometimes the only way to terminate filtering in the middle of the list of rules? We'll need some description of frequency to get the ++. Needs the review and super review. Once those are in, change the status to rtm+.
Whiteboard: [rtm need info]5587155871
yes, this is the only way to make filter execution stop without moving a message, set up a filter that moves messages to your inbox. It's a well known technique - I know because of the number of people who complained when I broke it. That combined with the fact that you lose the messages make it seem to me that we should fix this, since we have a trivial, low risk fix.
r=sspitzer,sr=alecf,mscott
Whiteboard: [rtm need info]5587155871 → [rtm++]5587155871
David: For QA verification purposes, could you reiterate the exact problem which was fixed here? Was it merely the move to inbox or a combination with such a move?
I realize the fix is not in yet, but enlighten me anyway...
Changing to rtm+, David inadvertently added one too many plus characters :-)
Whiteboard: [rtm++]5587155871 → [rtm+]5587155871
whoops, I meant to add 3 +'s! :-) thanks for catching that - I would have been waiting for the ++ bugsplat notification for a long time! laurel, it's simply a filter that moves to your inbox - that's all you need to cause the problem.
rtm++, pdt approved! :-)
Whiteboard: [rtm+]5587155871 → [rtm++]5587155871
fix checked in.
Status: ASSIGNED → RESOLVED
Closed: 24 years ago
Resolution: --- → FIXED
Still verifying, but here's a progress update using oct13 commercial branch... Linux: Looks ok on newly migrated profiles, did not see the problem. When I tried with today's branch on a profile for which I saw the problem in yesterday's build (i.e. I did not remigrate the profile) I continued to have a problem even after removing msf files... Will give it a few more tries, but wanted to report the difficulty. David, is there a specific way to clean up this problem on existing profiles other than having to remigrate/create new profile?
To clarify my last comments, when I sent/received New messages I did not get them on the older profile, I wasn't still trying for the older messages.
Hmmm... just noticed the date/time you said the fix was checked in... maybe I'm verifying for nothing (maybe the fix isn't in today), although I'm not seeing the problem with today's commercial branch builds (tried mac, NT, linux). Will look at again Monday.
the fix only affects newly retrieved messages. If you have an existing profile that lost messages, deleteing the msf file is not going to help - they're gone.
Yes, I figured that, but the problems I saw on Friday were that I couldn't get new messages on the profile which had exhibited the problem in previous build. Like I said, I'll try again on today/tomorrow's builds after we finish our basic functional testing.
OK using oct 18 mn6 branch commercial builds, linux rh6.0, Win98 and Mac OS 9.0 OK for new profiles, new accounts and newly migrated profiles (migrated with oct18th build). Will test further with older profiles having exhibited the problem previous to the fix to see if I can recreate the problems I described from last Friday. If I find anything I will report back. Will go ahead and considered this verified on the branch, adding vtrunk keyword for trunk build verification.
Keywords: vtrunk
*** Bug 52034 has been marked as a duplicate of this bug. ***
OK using dec22 trunk commericial build, linux rh6.0, win98, mac OS9.0
Status: RESOLVED → VERIFIED
Keywords: vtrunk
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: