User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1 Build Identifier: 2.0.0.18 (20081105) When sending mail with an .xls Excel file as an attachment, 14 of my client's 60 users are experiencing a problem whereby attempts to send Excel files as attachments fail to produce a message that can be decoded on the receiving side. The Content-Type of the attachment in the sent mail is incorrectly specified as "text/x-moz-deleted" and the file is not readable (by Thunderbird or other mail clients) on the receiving end. Instead, gibberish text appears in the message body (and it looks suspiciously like the binary contents of the file, rendered as text). After some debugging, we figured out that the affected machines have Thunderbird user profiles where the mimeTypes.rdf file contains (among whatever else) this text: ---- <RDF:Description RDF:about="urn:mimetype:externalApplication:application/vnd.ms-excel" NC:prettyName="" NC:path="" /> <RDF:Description RDF:about="urn:mimetype:text/x-moz-deleted" NC:fileExtensions="xls" NC:description="Microsoft Excel ワークシート" NC:value="text/x-moz-deleted" NC:editable="true"> <NC:handlerProp RDF:resource="urn:mimetype:handler:text/x-moz-deleted"/> </RDF:Description> ----- This apparently causes Thunderbird to use "text/x-moz-deleted" when sending .xls files as attachments, thereby causing the recipient to see them as gibberish. The unknown question is, why does this happen? (Why does the mimeTypes.rdf file get modified in this incorrect way?) I dont have any insight about that, but it is a pretty bad bug in the real world for businesses (like this one) trying to use Thunderbird as their main email client. Mailing Excel spreadsheets back and forth is extremely common among employees. When this problem hits, end users generally cannot figure it out. Additional notes: All the clients PCs we have here are running Windows XP Professional Japanese. None of the clients have any non-default add-ons installed. Reproducible: Sometimes Steps to Reproduce: Unfortunately, this only happens on 14 of about 60 client PCs we have. For the affected machines, it happens every time, and the repro steps are: 1. Create a new message via toolbar or menu command 2. Enter a subject line 3. Enter some text 4. Attach an Excel .xls file to the message 5. Send the mail. Actual Results: The attachment is sent with "text/x-moz-deleted" Content-Type. For the recipient, if they use Thunderbird, this results in a bunch of gibberish text at the bottom of the window. Expected Results: The attachment should be sent with some kind of appropriate Content-Type, e.g. application/vnd.ms-excel or perhaps application/octet-stream. Then at least the recipient could see that a file was attached, even if they did not have Excel. Unfortunately, I don't have good info or guess about why this affects some machines and not others. Initially, I suspected that the Windows XP Professional Japanese operating systems settings were somehow fubar'd, and so however Thunderbird was getting the mime type was yielding incorrect results. The fact that I found the problem in the mimeTypes.rdf file pretty much rules that out. (I also looked at the registry on the affected machines, and all is normal.)

This is an example of the borked mimeTypes.rdf from one of the affected machines here.

Such RDF entry is created when user tries to open attachment of filename="???.xls" and Content-type:text/x-moz-deleted using MS Excel. And there is no other way than "dialog when open" to create such entry, if standard Tb. (Such definition is seen via Tools/Options/Attachments/View&Edit Actions) However, Tb never propose "open" option(save/delete/detach too) to user if Content-type:text/x-moz-deleted, because Content-type:text/x-moz-deleted is for "deleted attachment". So such RDF entry can not be generated by current Tb, even if spammer sent attachment of Content-type:text/x-moz-deleted. When(with which version of Tb) did your user start to use Thunderbird? (RDF entry may be generated due to bug of old Tb) Does your user use extension(add-on) which relates to mime type/RDF entry handling? (Headers created by Tb for deleted attachment part) > --------------080103050404010901000400 > Content-Type: text/x-moz-deleted; name="Deleted: abc.xls" > Content-Transfer-Encoding: 8bit > Content-Disposition: inline; filename="Deleted:abc.xls" > X-Mozilla-Altered: AttachmentDeleted; date="Thu Dec 25 14:30:54 2008" > > The original MIME headers for this attachment are: > Content-Type: application/vnd.ms-excel; name="abc.xls" > Content-Transfer-Encoding: base64 > Content-Disposition: attachment; filename="abc.xls" > --------------080103050404010901000400 >(next part follows) > Expected Results: > The attachment should be sent with some kind of appropriate Content-Type, > e.g. application/vnd.ms-excel or perhaps application/octet-stream. Solution is very simple: Remove such garbages in mimeTypes.rdf. However, Fx/Tb removed "new/delete action in Helper Applications of Mozilla/Seamonkey" when Fx/Tb started to use "Tools/Options/Attachments/View&Edit Actions". AFAIR, there are some add-ons which can do it, but I don't know add-on name.

(In reply to comment #2) > Such RDF entry is created when user tries to open attachment of > filename="???.xls" and Content-type:text/x-moz-deleted using MS Excel. And > there is no other way than "dialog when open" to create such entry, if standard > Tb. Thank you for the answer. This is indeed how these entries are being created. > However, Tb never propose "open" option(save/delete/detach too) to user if > Content-type:text/x-moz-deleted, because Content-type:text/x-moz-deleted is for > "deleted attachment". So such RDF entry can not be generated by current Tb, > even if spammer sent attachment of Content-type:text/x-moz-deleted. This is not correct, at least not here on Windows XP. The attachment-related menu command are disabled, but double-clicking an .xls attachment received with the bogus Content-Type will cause the normal dialog for opening the attachment to appear. If the user chooses "Open with Excel" and checks the box to do this from now on, the bogus entry will be created in mimeTypes.rdf. I can now cause my freshly installed Thunderbird 2.0.0.18 to generate this bogus entry in mimeTypes.rdf whenever I want. Moreover, I can intentionally break other people's Thunderbird by sending them these bogus attachments. (By "break their Thunderbird" I mean cause their Thunderbird to start attaching the bogus "text/x-moz-deleted" to outgoing attachments.) > When(with which version of Tb) did your user start to use Thunderbird? > (RDF entry may be generated due to bug of old Tb) I don't know, as I have only been here for a few months. However, as I noted, I can make a freshly installed Thunderbird 2.0.0.18 generate this entry, now that I know how to do it. > Does your user use extension(add-on) which relates to mime type/RDF entry > handling? No, none of our users currently use any non-default extensions. > Solution is very simple: Remove such garbages in mimeTypes.rdf. This is a temporary fix, but the bogus entry in mimeTypes.rdf may be created again when the user tries to open one of these attachments. Thanks again for getting involved with this bug. I have to run to some meetings now, but I will post some more general observations I have a collected within a few hours.

(In reply to comment #3) > This is not correct, at least not here on Windows XP. The attachment-related > menu command are disabled, but double-clicking an .xls attachment received with > the bogus Content-Type will cause the normal dialog for opening the attachment > to appear. I could re-produce problem by double-clicking with Tb 2.0.0.18 and Tb trunk 3008-12-21 build on MS Win-XP SP3. Dialog was opened. Sigh... Possibly already reported problem. I'll try to search. => Confirming. > > Solution is very simple: Remove such garbages in mimeTypes.rdf. > This is a temporary fix, but the bogus entry in mimeTypes.rdf may be created > again when the user tries to open one of these attachments. There is no way to avoid that bogus entry due to this bug in mimeTypes.rdf may be created again, because it's impossible to inhibit use of double-click. Phenomenon/issues/problems after such bogus entry in mimeTypes.rdf is summarized in MozillaZine Knowlede Base. For example; (a) http://kb.mozillazine.org/MimeTypes.rdf (b) http://kb.mozillazine.org/Actions_for_attachment_file_types "External links" section of (b) may help you. > * The MIME Edit extension lets you add a new action, it is not limited to just editing and deleting existing actions.

FYI. Bug 417590 replaced "Helper Applications" of Seamonkey by "Tools/Options/Attachments/View&Edit Actions" of Tb. It means loss of add/delete functionality, and it means Tb's issue of "no way to recover from problem such as this bug" is imported to Seamonkey. So Bug 462629 is opened for Seamonkey.

More accurate reproduction steps: (This info is from notes I had typed up before replying to Comment #2, so there is a bit of redundancy.) OK, I have some new information today. This bug has spread from machine to machine in our office, much like a virus. A month ago I had one strange user who couldn't send Excel files. Pretty soon I had heard a few similar reports, yesterday I had 14 and today I have 20 users who can't send attachments. The key thing is, once a non-affected user tries to open a bogus attachment sent with Content-Type "text/x-moz-deleted", their own Thunderbird instance creates a new entry in mimeTypes.rdf, which subsequently causes their machine to become affected by the bug. That's because the modification to mimeTypes.rdf causes the .xls attachments they *send* to now have this bogus Content-Type. I think I have better reproduction steps now than in my original bug report, and can change the reproducibility to "always" instead of sometimes. --- 1) From Thunderbird instance A, find a message (send yourself one if necessary) with an .xls attachment. Open the message and delete the attachment. 2) Forward the message with he deleted attachment to Thunderbird instance B. 3) In Thunderbird instance B, double-click the attachment in the received mail. The attachment will have type "text/x-moz-deleted". 4) In the resulting open prompt, choos "Open with Microsoft Office Excel (default)" (assuming you have it installed), and check the "Do this automatically for files like this from now on". (At this point, Thunderbird B creates a bogus entry in mimeTypes.rdf that associates the ".xls" file extension with mimetype "text/x-moz-deleted.) 5) Now Thunderbird B is borked. It will now send .xls attachments with mimetype "text/x-moz-deleted", and this causes many mailers to see the attachment as gibberish inline text. Thunderbird, in particular, will not only be unable to display the message properly, but if the recipient does the same actions as step 3 above, their Thunderbird will now also be affected in the same way. --- Now, I think this bug has security implications, too, because a malicious user can send a Thunderbird user an evil message to intentionally break the recipient's ability to subsequently send attachments. Also, the virus-like way it spreads from machine to machine freaks people out. Some of my users were sure they must have contracted some undetectable virus. UNKNOWNS: 1.) I don't know what implications there are if the user already has a rule defined for .xls. I *think* they are still vulnerable, because the different Content-Type causes Thunderbird to prompt again for the new bogus attachment, but have not had time to test this. WORKAROUNDS: 1.) Editing mimeTypes.rdf helps temporarily, but the problem seems to recur. 2.) Installing the Mime-Edit add-on (v 0.6.0 as of today) does seem to fix the problem, if you manually add a rule for .xls. However, I have not yet had time to extensively test whether the problem can still reoccur. SUGGESTIONS: 1.) I think Thunderbird should anyway NEVER send an outgoing attachment with Content-Type: text/x-moz-deleted. That does not seem to me that it would ever be a valid thing to do. Instead, perhaps behave like Apple Mail which inserts some text something like [Attachment FooBar was deleted], in place of an attachment that has ben deleted in an outgoing message. 2.) Thunderbird should not modify mimeTypes.rdf to create entries for "text/x-moz-deleted". It seems perhaps a bit inelegant to have special-case behavior like this, but I think this is indeed a special case. There is never a time when a user wants this behavior. 3.) When double-clicking a received attachment with Content-Type text/x-moz-deleted, what should it do? This is a hard one because there are already vaild attachments in the wild, which unfortunately have this bogus Content-Type but do contain perfectly legitimate data. I haven't had time to think **** it, but I think that if Thunderbird receives a valid binary attachment but with this Content-Type, it should probably behave as if it were application/octet-stream. (This implies that it could tell the difference between an actual deleted attachment, but that should be possible because an actual deleted attachment has different content from a not-really-deleted attachment that Thunderbird has given the bogus mime type. See [A] and [B] below.) [A]: Example beginning of bogus-mimetype-attachment. Note that it has no X-Mozilla-Altered and no original mime headers. --------------030601010609080300070105 Content-Type: text/x-moz-deleted; name="=?ISO-2022-JP?B?GyRCJUYlOSVIGyhCMS54bHM=?=" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename*0*=ISO-2022-JP''%1B%24%42%25%46%25%39%25%48%1B%28%42%31%2E%78%6C; filename*1*=%73 0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAPgADAP7/CQAGAAAAAAAAAAAAAAABAAAAAQAAAAAA (BINARY FILE CONTENTS CONTINUE) [B]: Example of an actual deleted attachment: > --------------080103050404010901000400 > Content-Type: text/x-moz-deleted; name="Deleted: abc.xls" > Content-Transfer-Encoding: 8bit > Content-Disposition: inline; filename="Deleted:abc.xls" > X-Mozilla-Altered: AttachmentDeleted; date="Thu Dec 25 14:30:54 2008" > > The original MIME headers for this attachment are: > Content-Type: application/vnd.ms-excel; name="abc.xls" > Content-Transfer-Encoding: base64 > Content-Disposition: attachment; filename="abc.xls" > --------------080103050404010901000400

I changed the Component of this bug to Security, since I had so much fun today exploiting this bug to break co-worker's Thunderbird installations. Seems to be a vector for malicious actors to harm end users.

FYI. Old opposite request is found (request before detach/delete feature). > Bug 80254 Double-clicking an attachment should ALWAYS bring up dialog > asking what to do (open / save), no matter what the extension is By the way, please isolate problem of this bug from problems after this bug. (a) Due to this bug, by double-click of attachment of a mail, user can easily generate bogus entry of mimetype:text/x-moz-deleted with any file extension. (b) Because of (a) & (c), cracker/spammers can easily attack Tb users. (c) If bogus entry in mimeTypes.rdf is created by some reasons, many problems, issues, unwanted_phenomenon occurs. Problem (c) is already known issue, as seen in MozillaZine Knowledge Base articles. Please note that main problem of this bug is (a), although crashers/spammers can easily utilize this bug because of (c). I believe both (a) and (c) should be resolved. - Even if problem like (a) exists, critical issue won't occur, if easy way to escape from situation of (c) is available. - Even if problem like (c) exists, critical issue won't occur frequently, if problem like (a) doesn't exist.

With Seamonkey trunk(2008/12/26 build), issue of above (a) was re-produced. Delete attachment, double-click of deleted attachment -> Dialog, then bogus entry of mimetype:text/x-moz-deleted is created However, following issue, one of above (c), couldn't be reproduce with popular file extension such as ".gif", "xls", with Seamonkey trunk. Upon mail composition, attachment data with Content-Tyep:text/x-moz-deleted is generated for ".xls" or ".gif". Seamonkey has predefined table of mime_type/file_extension internaly for many popular mime_types/file_extensions. This Seamonkey's internal definition seems to have higher priority than user defined mimeTypes.rdf entry.

Correction of previous comment. Sorry for spam. When bogus entry of mimetype:text/x-moz-deleted is created, Seamonky trunk also created mail with Content-Type:text/x-moz-deleted for ".xls" file. Same problem occurred with Seamonkey 1.1.12. Double-click of deleted attachment -> Dialog -> mimetype:text/x-moz-deleted for ".xls" in mimeTypes.rdf -> Content-Tye:text/x-moz-deleted for ".xls" However, "Helper Applications" of Seamonkey 1.1.2 clearly displays the bogus entry and mime-type/file-extension of the entry, and has "delete" capability. So user can easily remove bogus entry by himself via UI.

Core issue. Changed to Product=MaiNews Core/Component=Attachment.

We've been unable work around this issue at our location. That is, the IT staff knows how to *temporarily* fix the problem, but the issue keeps recurring. Our IT staff has performed this procedure on almost all client PCs we have: --- 1.) Install the "MIME Edit" add-on (version 0.6.0) 2.) Use the MIME Edit GUI to delete any bogus "text/x-moz-deleted" MIME type associated with the file extension "xls" 3.) Using MIME Edit, create a new MIME type entry associating "xls" with "application/vnd.ms-excel" --- In our testing, this worked, and even after we double-clicked a bogus "text/x-moz-deleted" Excel attachment, the client continued to use the correct "application/vnd.ms-excel" content type when sending outgoing mail with Excel file attachments. So we thought we could work around this issue by modifying each client installation of Thunderbird in this manner. However, somehow, the problem is cropping up again on some user's machines. I am not sure of the mechanism. I mean, I am not sure what they are doing that causes Thunderbird to break again. Again deleting the bogus "text/x-moz-deleted" entry in MIME Edit does again temporarily fix the problem. However, a key problem is that the sender never notices that Thunderbird is sending bogus attachments; the problem is only apparent on the receiving end. So users end up sending bogus attachments for days before somebody finally complains to them. Right now I don't see any way to work around this problem effectively.

One interesting tidbit is that, as best we can tell, this problem started with a single user using the "Delete" contextual menu command on a single attachment, and then later forwarding that message. (That's the reason we only have this problem with Excel attachments.) The problem slowly spread (via email) from machine to machine after that, and wasn't noticed until about 20 Thunderbird installations were affected.

Followings are solutions that I believe currently exist on the Earth; 1. Emperor of your company bans "use of delete of mail attachment" 2. Emperor of your company bans "use of double-click of mail attachment" 3. Emperor of your company bans "use of Thunderbird" "Read Only" attribute of "mimeTypes.rdf" prohibited update of mimeTypes.rdf content by Thunderbird. (Checked with Tb trunk on MS Win-XP SP3) Following can be a workaround, if your company can accept "no additional relationship between mime-type and application". (1) Create null mimeTypes.rdf - Shutdown Tb, delete mimeTypes.rdf - Restart Tb, Tools/Options/Attachments/View&Edit Actions, Close panel => clean mimeTypes.rdf is created (2) Copy mimeTypes.rdf to common place, and set it to "Read Only" (Via Windows Explorer, ATTR mimeTypes.rdf +R, etc.) (3) Terminate Tb of each client's PC, and replace mimeTypes.rdf by one with +R. If standard/common "additional relationships between mime-types and application" is required, modify step (1) as follows. (1x-A) Create null mimeTypes.rdf - Shutdown Tb, delete mimeTypes.rdf - Restart Tb, Tools/Options/Attachments/View&Edit Actions, Close panel => clean mimeTypes.rdf is created (1x-B) Create additional entries in mimeTypes.rdf - Put mails which have attachment of required mime-type in mail folder - Double Click an attachment, and choose required standard action I believe; "loss by above workaround" << "gain by reduced claims from mail recipients" If a user deletes his mimeTypes.rdf, or if user removes "Read Only" attribute of his mimeTypes.rdf, problem can occur again. So another protection like next may be required. (1) Keep common mimeTypes.rdf(Read Only) on file server (say X:\mimeTypes.rdf) (2) Use BAT on each PC to start Thunderbird (say Y:\TB.BAT) - Copy X:\mimeTypes.rdf to all profile directories for Tb on the PC - Start Thunderbird Even if user created bogus mimeTypes.rdf in his PC, it'll be removed when user restarts Thunderbird again using the BAT file. So I think damage can be minimized.

Correction(sorry for spam). I misunderstood my test result. "Read Only attribute" could do nothing. Sorry but I can't find other protection than "Copy of clean mimeTypes.rdf before restart of Tb".

Hello, Thanks for the feedback, I appreciate it. (And, sorry this comment is so verbose, but I rarely get time to update this and so I want to try to include everything.) Unfortunately, none of the suggestions in Comment #15 really solves the problem. (Well, "stop using Thunderbird" would work, but my goal is to let the users use Thunderbird, while having sending email with attachments work. So I don't really count that as a "solution".) Even the batch file idea doesn't fix the problem until users quit Thunderbird, which they generally don't do all day. The way I see it, there are different elements of this bug. In order of simplicity, they are: 1.) Thunderbird should never, ever send an attachment file with content-type "text/x-moz-deleted", no matter what. This is always incorrect behavior, and fixing this should be trivial to code. (For clarity: I mean, when the user attaches a file to an outgoing message, there should be no case where text/x-moz-deleted is used. Even if the user has a broken mimeTypes.rdf, the mail sending code should special case this situation and use application/octet-stream instead for any outgoing messages with real attachments. I think the special case is justified here since text/x-moz-deleted is itself a special case, a magical content-type made up for Thunderbird that should never apply to outgoing messages with valid attached files.) Now at this particular office, where I first encountered this bug, we have modified every PC with Thunderbird (about 80), installed Mime-Edit, and temporarily fixed the problem. But, because this problem keeps spreading via email, and coming back and spreading again by email, I think at this particular office the decision has been made to replace Thunderbird with Outlook (yuck). It might be too late in the case of this office, but this scenario could happen in any office. So at a minimum, in my opinion, this aspect of the bug should be fixed with high priority. This aspect of Thunderbird's current behavior is unambigously incorrect. Actually, I don't think fixing this would be a *complete* solution (see below), but if this aspect alone was fixed, it would prevent thus problem from spreading to every Thunderbird installation on the network. When this happens, it really does make Thunderbird essentially non-viable as a business email solution, and all it takes to trigger the process is a single user deleting an attachment and then later forwarding that mail. A few lines of code would do the job, one would think. Even if implementing a special-case for text/x-moz-deleted is not an elegant and complete solution, it would ameliorate the catastrophic aspect of this bug. 2.) The way Thunderbird handles deleting attachments is, in my opinion, also fairly broken. I think more correct behavior is implemented by Apple's Mail program. I append an example below. I think there is no reason to keep the "filename" portion of the attachment, if the attachment is deleted and there really is no longer a file. Apple's way mentions the filename in the text that replaces the attachment. I think with Apple's way, there is very low chance to misinterpret the results, as Thunderbird does. That misinterpretation is what causes Thunderbird to associate text/x-moz-deleted with files that are actually valid attachments. I cannot say Thunderbird's current behavior is "always unambiguously incorrect behavior" as I can with #1 above, but I think it is incorrect. Fixing this aspect might be more difficult. 3.) Finally, the whole way Thunderbird handles content-types/mime-types causes a lot of problems, and not only this catastrophic bug. I understand that Thunderbird is inheriting code from Firefox, but a mail client has more complex needs in this regard than a web browser. For example, just because I choose to open my .doc attachments with OpenOffice (I don't, but for example's sake), does not mean I want them to be sent to other people with content-type application/vnd.sun.xml.writer. Thunderbird's current behavior for associating MIME types with documents is inflexible, does not work too well, and has no mechanism for the end user to control or adjust it. I think simple is great, but in this case some mechanism for controlling this is actually important for most users, to get correct/optimal behavior. Now, this aspect is totally convoluted, and perhaps even political, and there is probably no unambiguous solution that would be agreed to by all. Even just figuring out how to improve this aspect would be a lot of work, without even starting to code. I personally think that a fairly comprehensive list of hardcoded default mime-types for outgoing messages would be better than the current situation. But I am sure there are already a bunch of bugs related to this, so I think it is outside the scope of this bug report. This is my first time supporting Thunderbird, and so I am not that familiar with the project and how you guys process and triage bugs, but my own idea would be to narrow the scope of this bug report down to #1 above, because it is the simplest unit of broken behavior that is causing the actual catastrophic failure to occur, and the fix (while arguably inelegant) is easy and unambiguous. I would hope that #1 gets fixed very soon, rather than getting deferred until there is time to implement a more sweeping and beautiful fix that also encompasses 2 and/or 3. Cheers. --------- examples cited above follow ---------- Apple Mail's way of removing attachments removes the filename, since there is no longer a file: --Apple-Mail-2-973171880 Content-Disposition: inline Content-Type: text/plain; charset=US-ASCII [The attachment p4.pdf has been manually removed] --Apple-Mail-2-973171880 ------------------------ Thunderbird's way includes the filename, which I think is non-optimal: --------------090300080403070702050507 Content-Type: text/x-moz-deleted; name="Deleted: test_mason.xls" Content-Transfer-Encoding: 8bit Content-Disposition: inline; filename="Deleted:test_mason.xls" X-Mozilla-Altered: AttachmentDeleted; date="Wed Jan 28 10:53:54 2009" The original MIME headers for this attachment are: Content-Type: application/vnd.ms-excel; name="test_mason.xls" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="test_mason.xls" --------------090300080403070702050507--

I have determined that all a user has to do to re-break their Thunderbird (that is, cause this problem to manifest again even after it has been manually fixed) is to go find an old message containing one of these "text/x-moz-deleted" attachments and double click the attachment again. This re-borks their mimeTypes.rdf file, and they are once again unable to send attachments correctly. I don't know why our testing seemed to show that installing MIME-Edit prevented that, but I was apparently wrong.

There is one other way to fix the worst impact of this bug, which also is limited in scope and only involves fixing an aspect of Thunderbird's current behavior that is unambiguously incorrect. If the code that associates MIME types with attachments were modified to never associate "text/x-moz-deleted" with valid attachment files, that would prevent the most heinous problem. That is to say, users who open incoming email attachments still might have the MIME type for a given type of file set to something other than what they would want, but: a.) it would no longer break their Thunderbird in such a way that they become unable to send email with attachments b.) the damage would no longer spread from machine to machine via email This might be even simpler to implement than item 1 from comment #16. If this were implemented, Thunderbird installations which currently have borked mimeTypes.rdf would still each need to be repaired (have mimeTypes.rdf fixed or deleted), but the problem would no longer reoccur after that was done.

Correction to comment #17: To re-bork their Thunderbird, the user must not only double-click a corrupted attachment (one with the incorrect content-type of "text/x-moz-deleted"), but also check the "Do this automatically for files like this from now on" box.

(In reply to comment #19) > To re-bork their Thunderbird, the user must not only double-click a corrupted > attachment (one with the incorrect content-type of "text/x-moz-deleted"), but > also check the "Do this automatically for files like this from now on" box. You are right. When I simply tried to open the "text/x-moz-deleted" by an external application, bogus entry was not generated. Checking of "Do this automatically for files like this from now on" was required to generate bogus mimeTypes.rdf entry. It's the reason why I misunderstood my test results and was confused many times. There are three major kinds of problem. (1) Open dialog easily generates bogus mimeTypes.rdf entry. (1-A) If spammers or mailer with incorrect set-up sends mail with attachment of wrong relation between mime-type & file extention, Tb's open dialog via menu of double-click can generate bogus mimeTypes.rdf entry. (1-B) Tb permits Tb user to generate corrupted mimeTypes.rdf by invoking open dialog box even for "deleted attachment" created by Tb himself. Situation is worse than (1-A), because Tb is main culprit in this case. Note: (1-A) is already known issue which is not resolved for long duration. (1-B) is new problem discovered by you. (2) Tb's Tools/Options/Attachments/View&Edit Actions can do nothing on already generated bogus entry in mimeTypes.rdf. When bogus entry of "text/x-moz-deleted" case, information relates to the bogus entry is not displayed in "View&Edit Actions" panel of Tb. Note: It's also already known issue of Tb for long time. Bug 462629 is request for Semonkey trunk. It's request not to import Tb's problem of (2) to Seamonkey trunk. (as I said, Seamonkey 1.x doesn't have problem of (2).) (3) User can easily send mail with incorrect mime-type. (3-A) Once bogus entry(wrong relation between mime-types and file extension) is generated in mimeTypes.rdf, Tb user can send mail of (1-A) or (1-B), being oblivious of wrong mime-type. (3-B) When problem of (1-B), situation is much worse. (your case) User can easily propagate mail of (1-B) to many co-workers by forwarding his mail with "deleted attachment". Note: It's also already known issue of Tb and Seamonkey. Please note that new discovery by this bug is (1-B) only. All other problems you are experiencing is already known issue which will occur once bogus mimeTypes.rdf is generated.

Yes, not checking the "do this automatically" box is the reason I also was confused by my testing results. I understand what you mean about 1-B (and therefore 3-B) being the unique discovery of this bug report. As you say, 3-B is a much more serious problem than any of the others. Bogus MIME types can be annoying but it is not usually a terrible problem. Normally, the attachment might still be usable. (In the case of "text/x-moz-deleted" it is worse than usual, because the recipient cannot open the attachment, if they also use Thunderbird.) However, because of 3-B, once one single user has triggered this bug, his entire company or other organization using Thunderbird can be broken. In the company I am trying to help, almost all the Thunderbird installations (about 80 PCs) eventually became unable to send attachments. The bug propagated by email, and because of old emails, it happened again and again, even after the IT staff fixed the mimeType.rdf files. Obviously, a bug that can spread from one machine to another via email and afflict an entire organization is more serious than a bug that does not spread.

Can your customers introduce next procedures? (a) Start Tb with BAT (copy clean mimeTypes.rdf before restart of Tb) (b) Shutdown Tb after end of daily Job, and restart Tb upon start of next day. I believe (b) is normal/usual practice in business environment. I can't believe that every employee of your customers does do double-click of attachment every an hour. I can't believe that the every employee who created bogus entry in mimeTypes.rdf will send mail with attachment of the bogus entry evry an hour. If (a) & (b) is possible, problem on one client PC will disappear within 8 hours on the PC. If so, I think frequency of the biggest issue of "propagation of problem" will become very low.

By the way, I recommend you to open separate bug for problem of (1-B) only. I think this bug is slightly messy for problem (1-B), because too many comments are already written for "problems on Tb users after (1-A) or (1-B)".

> Can your customers introduce next procedures? > (a) Start Tb with BAT (copy clean mimeTypes.rdf before restart of Tb) > (b) Shutdown Tb after end of daily Job, and restart Tb upon start of next day. Thanks. In this company, people really do email each other Excel files quite often. There are many standard documents and report forms they use, which are all in.xls format. And the basic way they submit reports is by email. I did some quick analysis of ten machines, and almost 50% of emails sent have attachments, about 80% of which are Excel files. Maybe this is unusual, I'm not really sure. I have a few clients using Thunderbird, but this company is the only place I have seen this problem so far. Anyway, they already do (b) (shutdown PC at end of day), I did something very similar to (a) the .BAT file idea[1], but the problem does still recur. And it spreads very fast, because often one excel file is sent to, say, 10 people. If that happens at 10AM, and every recipient opens the attachment and borks their Thunderbird, and each of those people sends reports to 3 other people in the next 5 hours, by 3:00 PM there are 40 Thunderbird installations which are broken again, even with daily restarts that nuke mimeTypes.rdf. I believe one problem is that there are now hundreds or even thousands of old emails, which contain these bogus attachment files. It initially took several months for this problem to spread from one user to many users. By the time the problem was noticed, and reported to me, there were already many many of these files. It seems that people often look up an old file in their email archive, rather than looking for it on the hard disk. And since this company is still using POP, these emails are spread across many PCs. So the only "solution" has been to start training the Thunderbird users to a.) Never use the "delete attachment" function of Thunderbird b.) Never send an Excel file as an attachment. We're training people to zip every Excel file before sending it as an attachment. They strongly dislike this, though, because it adds work to a task they do many times per day. (Also, it is worth noting again that it just happens to be Excel files at this company. It could be any file type.) So, the choices are really only: a.) Fix Thunderbird to prevent this problem from happening or b.) Stop using Thunderbird Through this discussion, I have gotten some ideas for how to fix Thunderbird, so I have asked for another few days to see if it is viable. --- [1] What I did, which has the same effect as the .BAT file idea, is replace mimeTypes.rdf with a folder instead of a file. This causes Thunderbird to be unable to replace mimeTypes.rdf with the bogus text/x-moz-deleted data. However, after opening a borked attachment, the problem does remain until Thunderbird is quit and restarted.

(In reply to comment #24) > almost 50% of emails sent have attachments, about 80% of which are Excel files. > I have a few clients using Thunderbird, but this company is the only place I have seen this problem so far. > Anyway, they already do (b) (shutdown PC at end of day), I did something very > similar to (a) the .BAT file idea[1], but the problem does still recur. Oh I see. I now understand why this bug's issue(1-B) will produce so-severe problem so-frequently/so-repeatedly on one of your customers. I recommend you again to open separate bug for (1-B) ASAP, with severity=critical or major(critical:user's view. major:programing view), with keyword of "regression", referencing to this bug. Another way: Request charged support to mozillamessaging.com, if avail. > http://www.mozillamessaging.com/en-US/

(In reply to comment #23) > By the way, I recommend you to open separate bug for problem of (1-B) only. I > think this bug is slightly messy for problem (1-B), because too many comments > are already written for "problems on Tb users after (1-A) or (1-B)". Thanks. I agree, it is too messy, and my understanding of the problem(s) is much better now than at the beginning when I filed this bug. So, I will file new bug reports. I think there are several different bugs involved in this problem. There are already several existing bugs about how Thunderbird's handling of MIME types has some problems. But also, if any one of the following three bugs were fixed, this catastrophic situation would not have occurred: --- 1. Thunderbird should never send a real attachment with content type "text/x-moz-deleted". This behavior is always incorrect. There may be many places that Thunderbird asks for the MIME type of a given file: mimeTypes.rdf, the OS, or some other entity. However, if the answer is "text/x-moz-deleted", Thunderbird should ignore that and use "application/octet-stream" instead. It is not possible that "text/x-moz-deleted" would ever be correct for a real attachment, and since this magic MIME type is a special case that changes Thunderbird's behavior on the receiving side, it causes serious harm when this mistake occurs. There are now other bugs which cause this to happen, and there could be future bugs which cause it to happen again. So, in my opinion, the code that encodes the message content for sending is the right place to ensure this error never occurs. Defensive coding here would prevent other bugs, which normally would be minor, from having seriously negative effects. 2. Thunderbird should never associate MIME type "text/x-moz-deleted" with any type of file. This behavior is also always incorrect. At least for this special case, there should be a sanity-check before the MIME type is associated with a file type (file extension), and this always-wrong behavior should be prevented there. Now, perhaps this is just a more serious example of an already-existing bug, so instead of a new bug I should just find the existing bug, and add a comment to it. But in the special case of text/x-moz-deleted, the result of associating an incorrect MIME type with a file extension has much more serious consequences. 3. Thunderbird's mechanism for "delete attachment" and "detach attachment" is a bit broken. On this one, unlike 1 and 2 above, I am not 100% sure that the current behavior is incorrect, but I think it is. As I said in comment #16, there does not seem to be a valid reason to include the "filename" parameter in the replacement for the deleted attachment. (Is it called a "MIME part"? I am unsure of terminology.) There is no longer any file; therefore, there is no longer any filename. The file is replaced by informational text, to inform the user of what was deleted. This text should (and does) inform the user of the name of the file that was deleted, and other info. But, it should not have the filename parameter which causes Thunderbird to think there is actually a file attached (and therefore, to trigger the incorrect behavior). I think the Apple Mail example in comment 16 is a more correct way to implement deletion of attachments. --- So, if any one of these three issues were fixed, it would have prevented the Thunderbird breakage, and propagation via email, that I have seen in this case. However, I do think all three of the issues should be fixed. I will do some searching of existing bugs and then file appropriate bug reports.

One more note: I created a proof-of-concept patch, and built yesterday's Thunderbird 3 source code (Thunderbird 3.0b2pre) on Windows XP, and fixed the problem. I am mainly an Objective-C Mac programmer, not a C++ Windows programmer, and until yesterday had no experience with the Mozilla/Thunderbird code base, so it took me most of the day to figure out how to build Thunderbird on Windows and then find the right place in the code. I modified nsMsgComposeAndSend::AddCompFieldLocalAttachments() in nsMsgSend.cpp, to prevent Thunderbird from ever sending attachments with text/x-moz-deleted (and have it use application/octet-stream instead). My patch is not yet production-quality, but I demonstrated that this problem can be fixed if issue 1 from comment 26 is addressed. Of course, if I can get the patch into good enough shape, I will submit it. One problem is that I don't yet know how to tell whether an attachment is real, or whether the user really is forwarding a message from which they have deleted attachments, and so even in the latter case, it uses application/octet-stream. This is not correct, and causes actually-deleted attachments to display to the recipient as if they were real attachments. On the other hand, due to issue 2 of comment #26, the "correct" behavior causes much worse problem for the recipient (i.e., breaks their Thunderbird if they double-click the attachment). So for now this incorrect behavior is better. Using a build of Thunderbird 3 with my patch prevents this problem from recurring at this office, but I am not sure whether they are willing to run a custom version of Thunderbird or not. (Updates would become a hassle, etc.) In the best-case scenario, I'll be able to get this patch into decent shape and it will be accepted soon, and the problem at this office would be solved, even if the other issues remain. Or maybe even if my patch still kinda sucks, somebody with more Mozilla experience will be able to fix it up and get it integrated.

OK, I have created one new bug report, which is more specific: bug 476133 This is for issue 1 from comment #26. I haven't yet had time to search and refine bug reports for issues 2 and 3 from comment #26. It is Friday night here in Japan, so that will likely have to wait until Monday. I think bug 476133 represents the easiest way to fix the most serious aspect of the real world problem related to this set of bugs. Therefore I am marking this bug as RESOLVED. I chose INVALID since it seems the most appropriate resolution status (as in, the conversation on this bug is too convoluted to easily follow). It would be nice if sombody could accept bug 476133 so time isn't lost.

(In reply to comment #28) > I have created one new bug report, which is more specific: bug 476133 Why the first separate bug is not for problem of "dialog open by Tb when double-click of deleted attachment"? It's the root cause of bogus mimeTypes.rdf in your case and severe problems on your customer after it, isn't it? > I chose INVALID since it seems the most appropriate resolution status. Why this bug can be INVALID. All Tb's problems pointed by you in this bug really exist. This bug is simply not appropriated to handle a specific problem. (single problem per a bug is rule). I believe this bug is better to be kept as open, until we'll find appropriate bugs for already known problems which will occur after bogus mimeTypes.rdf entry. Reopening this bug, in order to keep this bug as temporary/pseudo meta bug for problems around bogus mimeTypes.rdf due to improper/inadequate design/implementation of Tb. (This bug is best example for asking to never import Tb's problem to Seamonkey Trunk :-) )

OK about reopening this bug. I think either 1 or 2 (from comment #26) could be considered the root cause. If TB did not send the broken message, this problem would not exist. Next, if TB did not respond badly to the broken message (by polluting mimeTypes.rdf), the problem would not exist. If 1 is fixed, Thunderbird will not break itself and spread the damage as a result of normal use. If 2 is fixed, this problem would not occur even if a malicious attacker tried to harm a Thunderbird user intentionally (by manually making this type of bad message). I strongly think both should be fixed, but I chose to file the bug for 1 first because it is the first bug to occur in the sequence, and also because I think it is very easy to fix. Anyway, I will also file bugs for 2 and 3, when I get to the office on Monday.

(In reply to comment #30) > If 1 is fixed, Thunderbird will not break itself and spread the damage as a result of normal use. Does it mean "delete attachment" itself is fortunately not popular in the victim company? Small number of peoples who love "delete attachment"+"double-click of it" initially delivered problem to many co-workers by sending mail?

Yes. In this company, sent mail is stored on each PC (still using POP, not IMAP), and so I was not able to audit every sincle PC, but after poking around I believe the entire problem began with one single user who used the "delete attachment" feature on some of his messages to delete ".xls" attachments, and then later forwarded one of those messages to some co-workers. Each co-worker who tried to open the now-bogus attachment suffered the pollution of mimeTypes.rdf, and then their Thunderbird was broken, so each person to whom they sent ".xls" attachments suffered the same fate and so on. Initially the problem spread slowly but once 15 or 20 Thunderbirds were affected, it became fast-spreading and difficult to eradicate. I cannot be 100% sure, but it seems the entire company-wide problem started withjust this one single email, sent after its attachment had been deleted by the user.

I have created a second bug report which is derived from this bug: bug 476400. Also, I have given up filing a bug for the way TB implements deleting attachments, because I cannot figure out how it should be done. I tested it out using Apple Mail's way, by manually editing some test mbox files in a TB profile, but even without the filename parameter in the replacement part, TB still tries to open the deleted attachment when it is double-clicked. So, I don't really have enough knowledge to make a good recommendation. Therefore, this bug has the following two descendents. Fixing either of them would prevent the damage-propapgates-by-email aspect of this bug. 1. bug 476133 "Incorrect x-moz-deleted MIME type for sent attachments harms recipient's Thunderbird, and then breakage propagates via email to every Thunderbird on the user's network" 2. bug 476400 "Double-clicking deleted attachment breaks ability to send attachments, and this breakage spreads to other Thunderbird users via email"

Since bug 476400 is now fixed, i'm not sure what this bug is about.