The client uses the string "From" internally to delimit messages within its mail database. In many cases, a ">" character gets inserted into the file if a line of message text begins with "From" to prevent the client from interpreting it as the beginning of a new message when reading from disk. In many cases - but not all - the > actually ends up being displayed to the user in the UI. This means the message is not shown as it was sent, but tampered with. This would be just a cosmetic (but annoying) problem, except for S/MIME signed messages. The tampering that occurs in the file when the message is stored on the disk can cause the signature check to fail, even if it is valid. I don't have a test case with Mozilla because I don't have a build with functional S/MIME currently, however I have a test case with Communicator. 1. go to Outlook (this is necessary so that no message tampering occurs) 2. import your cert 3. send yourself a signed message. Include just one line, without a carriage return, beginning with the word "From". My test message just contained the words "From Julien".4. read the message in your Netscape client. In Communicator, an "invalid signature" icon is displayed. If you do a view/source of the message, you see that the first line says ">From Julien". This is what causes the S/MIME signature check to fail. The fix for this is for the client to use a better mail storage method that does not tamper with the content of the messages. I have not checked this with the Mozilla / Netscape 6 S/MIME builds because again, I don't have one where the S/MIME UI is showing, but I know the mail storage is the same in Mozilla as it was in Communicator, and I expect this S/MIME bug to show up. Just today I got a non-S/MIME e-mail from bugzilla where I had written a sentence beginning with the word "From", and it got mangled as ">From" when I read it in Netscape 6.2 ...

Julien, Today's build work even though the menu do not show up. You can either: act as if the menu string were present: The option->[security] menu (string security missing) shows a sub menu. The sub menu has two part separtaded by a horizontal sep. the first part has three radio buttons. Make sure the second one is on for encrypt. The part below the separator is a check mark. check means to sign. The string may appear tomorrow.

I still don't have a functional S/MIME browser build. However, I'm now confident that this problem will show up in it. Just using the smime perl script from the toolkit to send a signed message containing the string "From Julien" in the body causes Netscape 6.2 to prepend a > to the From word. This will cause the signature check to fail.

How can I tell that the signature failed? I sent myself a signed message with "From Scott" in the body. I received the message and it shows up as ">From Scott". But I don't see any error message. The message displays and it says "signed" in the header area.

Scott, If the signature is not shown as invalid in my test case, it might be that the client is not verifying S/MIME signatures at all, but rather just reporting that the message has a "signed" attribute. If that's the case it would be extremely dangerous since it would show tampered messages as simply "signed" ...

Are we going to be verifying signatures?

Now Mozilla verifies the signature. But I don't know if the bug is still there. Note if you send an signed e-mail directly from Mozilla (with the From in the message on start of a line) it verifies fine, because Mozilla prepends a space before the from just before it signs the message. I don't have outlook so I can't send myself a signed message with From on beginning of line. If someone can send it from outlook to my email (tomas.mraz@pvt.cz) please do so.

I have just verified that the bug is indeed still there. I sent myself a signed e-mail from Outlook. The message was raw text and simply said on the first line "From Julien, with love" (without the quotes). I retrieved the message in Mozilla. It showed as ">From Julien, with love" . There was a broken pen icon, and when I clicked it, it said "Digital Signature Is Not Valid". I also retrieved the message in Communicator 4.79, and it also displayed as ">From Julien, with love" and got the "Invalid signature" icon in the upper right corner of the message window. I think this is a showstopper for S/MIME in Mach V. We can not rely on S/MIME working if our database knowingly corrupts the message by adding this > character to all lines that begin with the word From. The message database format must be modified in order for S/MIME to work. Setting priority as P1. Please set the target as I'm not familiar with mozilla schedules.

Bear in mind this only happens on plain text messages, so by default, Mach V won't send out messages that this happens to.

I am aware of this, but not everyone uses mozilla for mail ! A heck of a lot of people use Outlook. There are probably other S/MIME clients, and they may also send the signed messages plaintext. The S/MIME spec allows it, and Mozilla doesn't follow it, as it incorrectly considers some messages that have a perfectly valid signature to be tampered with.

I'm just trying to make sure everyone is aware of the facts. Also, we will not be changing our mailbox format for Mach V. It's way too dangerous. The only way I can imagine this getting fixed is to strip off the > before the From when passing it to whatever code it is that's trying to calculate the signature - that's the code that's going to need to change, IMO. I don't know how that code works, however, so I don't know about the feasibility of that (and how we'll handle messages that really have >From in them...)

David, That still won't work. Take the case of the message below. Assume it gets sent from Outlook to a mozilla user. >From what I read, X From what I read, Y The above message contains lines beginning with both >From and From . When the Mozilla user receives it, it gets stored in mozilla as : >From what I read, X >From what I read, Y If you then strip the > before the From it will become : From what I read, X From what I read, Y When you pass that to the signature verification code, the signature check will fail. The signature check can only pass if the message has not been tampered with between the time the message was signed and the time it is verified. Your suggestion to strip the > before From is an attempt to undo the tampering, unfortunately it doesn't work because at the time the message gets stored in the database, you don't know which lines got the > added before From and which ones already had it in the first place. A solution would be perhaps to add a special header in the message in the database, call it "Hack-From:". That header would have a list of every lines that got a > prepended. Then you would be able to strip only those lines of the > before doing the signature verification, and it would work. You could also do that stripping before displaying/forward/doing all kinds of other operations on the message, so that they no longer appear mangled, even though they are within the message store.

Oh, by the way, the message above *must* be read over HTTP on bugzilla, otherwise it makes no sense. The e-mail notification message gets mangled in Mozilla or Communicator because of this very bug ...

Here is suggestion how to make it work for ALL messages: Current behaviour: Sending plaintext message: Space is inserted before all lines which have 'From' at beginning NOT COUNTING SPACES. (I don't know if it's signed before or after inserting the space.) Receiving plaintext msg: '>' is inserted before all lines which immediately begin with From Verifying and displaying plaintext msg: One space is removed on all lines which begin with n-spaces and From. (Again I don't know if it's verified before or after removing the space but it's done so the signature is verified.) The space is removed only in e-mails sent from Mozilla - it isn't removed on mails from other clients but I don't know what's the trigger exactly. Suggested behavior: No mangling of messages when sending them, receiving them and so on. Storing into local mailbox: Insert space before lines which begin with From NOT COUNTING SPACES. Retrieving from local mailbox: Remove one space from beginning of lines which begin with From NOT COUNTING SPACES. The signing is done before storing into a mailbox and the verifying is done after retrieving it so it's done on unmangled messages and everything should work fine. It should be relatively easy to do, maybe the hardest work would be on the removing of the current partially working kludge solution. :-)

The problem with that is interoperability between versions - we don't know which version of Mozilla or Mozilla-derived products sent the mail. We could do a similar fix with escaping of From lines from outlook, except use > instead of space. So, if the line starts with 0 or more '>' followed by "From ", then add a '>', and when calculating the signature, if the line starts with 1 or more '>' followed by "From ", remove 1 '>'. This would only work for newly received mails that dealt with lines starting with >From but how often do users go back to old mail and verify signatures. Also, to be clear, this is only for POP3 messages and not IMAP messages.

Yes. We can leave the old mechanism there as is and add the mechanism I suggested but using different character - preferably >. This would allow 100% compatibility with older versions. The crucial question is if the current kludge mechanism adds the space before From before it makes the signature or after. I think that the first case is much more probable because it allows to verify the message fine in other S/MIME clients. And it then removes it only for display. So the only compatibility problem with older Mozilla releases when this mechanism would be removed would be that with a new Mozilla you would see the space before From. And in the old Mozilla you wouldn't verify the signature. But since Mozilla isn't end-user product and since there hasn't been yet released a product based on Mozilla which would use the S/MIME mail maybe it doesn't matter. How is the problem solved in NS4.x? I don't have it so maybe there could be other issues when communicating with NS4.x mail. Conclusion: We don't have to remove the old mechanism but it would be surely nice clean up. But what we need as soon as possible is to change the mechanism which adds the > character to add it in >....>From cases too and add the code to remove it before verifying and displaying the message. Could this make it to mozilla1.0? Probably not :-(

This didn't work in 4.x either, according to jpierre. I don't know where the "adding of the space" kludge came from, but I'm betting it happens before the signature is calculated, so it should be working fine. No, I don't think this will happen for Moz 1.0.

Yes, it was broken in 4.x, and that was an extremely embarassing S/MIME bug. Mozilla is the chance to finally fix it. Are we going to take it or do we just want to merely match what our previous 4-yr old release did ?

It's shame it didn't get attention a little bit sooner. Nominating for Mozilla 1.0.1

In this case we're going to merely match what our previous 4-yr old release did.

*** Bug 167232 has been marked as a duplicate of this bug. ***

My signature was " ---------------------------------------- From Mozilla and GNU/Linux " but I changed it to " ---------------------------------------- Sent from Mozilla and GNU/Linux " because someone knew about this bug. Yeah, I have a work around.

It seems that the escaping of lines starting with the word "From" is not required by Internet mail standards (<a href="http://www.faqs.org/rfcs/rfc822.html">RFC 822</a>). Therefore, escaping is not necessary or standard (although it is used by some programs, like currently Mozilla Mail for incoming messages and importing). For example, Eudora uses standard text mailboxes, but does not escape "From" lines in the message body, because clearly people don't like their "From" phrases transformed into ">From". (Personal note from a newcomer: In my opinion, Eudora is good in this point of no escaping, but of course Mozilla Mail do already better than Eudora in many other important points, like support for Asian languages, open source, constant improvement, etc. Now, I'm moving soon from years of use of Eudora to Mozilla.) A good and standard solution, with "From" start-of-message header lines but without quoting "From" lines in the message text, is described in the following link: University of Washington: IMAP Toolkit Frequently Asked Questions, 6.12 <a href="http://www.washington.edu/imap/IMAP-FAQs/#6.12">http://www.washington.edu/imap/IMAP-FAQs/#6.12</a>

In the case that the solution of the above link could take some programming time, a good temporary solution would be to return to the escaping with a space used in Mozilla Mail previously (converting "From " into " From " at the start of lines, in the message body). This space escaping (" Form ") works perfectly to avoid any confusion with the start-of-message "Form " lines, and therefore to avoid message fragmentation. You can verify it editing a message file in Mozilla Mail with a text editor. It seems that this was unfortunately changed to use ">From " about one year ago (see http://bugzilla.mozilla.org/show_bug.cgi?id=119441 ). I think this happened because import tools lacked any "From " escaping, so they added ">From " and later changed " From " in other parts of the program into ">From ", thinking it was more standard (it seems that it's not so: see RFCs 822, 976, 1521, 2822...). Probably it would be very easy and fast to implement the temporary solution, changing '>' into ' ' in the code for the "From " space escaping, until a complete solution for Mozilla's standard mbox mailboxes (regularizing the start-of-message "From " lines), such as the solution of the above link from the IMAP Information Center at the University of Washington, can be added later if approved.

Fixing bug 58308 could help this (maildir folders do not need >From escaping)