I finally got round to playing with ajaxSketch today, and unfortunately discovered what seems to be a rather serious problem in the way Mozilla saves MIME types it hasn't seen before. To provide the save functionality ajaxSketch sends the drawing with the MIME type "x-type/x-doc" to put up the "What should Firefox do with this file?" dialog. When Firefox saves the file it stores a record of this previously unencountered MIME type along with the filename extension .svg in mimeTypes.rdf (on MS Windows at least). It appears when you open a local file, Firefox always goes to mimeTypes.rdf to obtain the MIME type corresponding to the file's filename extension first. Only if mimeTypes.rdf doesn't have anything useful does it ask the OS for the MIME type. Unfortunately, after saving an ajaxSketch drawing, Firefox thinks that files with the .svg filename extension have the MIME type "x-type/x-doc" (because that's what's now in mimeTypes.rdf). Since Firefox doesn't know what to do with files of this type, it will now ALWAYS put up the "What should Firefox do with this file?" for local .svg files and will be unable to open them itself. The user will suddenly be unable to view SVG files that previously worked on their computer. I don't know how many people have given ajaxSketch a try, but I guess there are now considerably more people out there with a corrupt mimeTypes.rdf than before, and that the numbers will continue to increase until hish/michaelr change what ajaxSketch does. (I've mailed them explaining the problem and asking them to remove the Save functionality until they find another way to do it.) It seems clear to me that this is a bug in Mozilla. Maybe Firefox should go to the OS *before* looking in mimeTypes.rdf? Or maybe it shouldn't save stuff to mimeTypes.rdf if the OS already has an entry for the filename extension that would be stored with the MIME type? A secondary issue is what we're going to do with the people with a corrupt mimeTypes.rdf? Fixing the file manually would be a pain for people, and if there are enough of them, then maybe a small extension/bookmarklet could do the job automatically? Thoughts?

> It seems clear to me that this is a bug in Mozilla. Why? > Maybe Firefox should go to the OS *before* looking in mimeTypes.rdf? That would make it impossible to locally override OS associations. > Or maybe it shouldn't save stuff to mimeTypes.rdf if the OS already has an > entry for the filename extension that would be stored with the MIME type? That's an interesting idea, as long as we aren't saving more state. I believe this saving happens in the UI, not in core file handling, but I could be wrong. Past all that, imo the behavior of ajaxSketch here is a bug. Have you filed it on them?

Why? Well because it allows malicious people to trick Mozilla users into breaking the loading of local files they probably want Mozilla to load, and it allows non-malicious people to break things accidentally. Oh, and if anyone saves one of those image/svg-xml files out there, it will break too. It's very easy to break local SVG files for Mozilla it seems. > That would make it impossible to locally override OS associations. Not exactly. It would mean that *new* overrides wouldn't be added on the basis of a bogus MIME type served from *any old* website out there. Predefined overrides would still exist. Is there a real need to add arbitrary new ones? > That's an interesting idea, as long as we aren't saving more state. Hmm. The problem with only adding it if the OS doesn't have an entry is that users may not have encountered SVG (or whatever) yet, but say they start using that filetype at a future date, they will initially have to figure out why Mozilla won't load this type that it *should* support. It seems that this problem doesn't occur for .html files. My attempt to corrupt mimeTypes.rdf for .html failed. See: http://jwatt.org/tmp/mimeTypes.rdf.buster/break-local-html.html So I guess we're doing something special for .html, and should probably be doing the same thing for .svg and .svgz at least. I've emailed Michael Robertson and Hisham El-Emam directly about this problem. I didn't suppose they'd have an issue tracking system, so I didn't look for one.

> Is there a real need to add arbitrary new ones? Well, users need to be able to add them, yes. Perhaps the real problem is that our extension-to-tyoe database is the same as our general MIME handling database; I'd be fine with that changing, but we'd need to migrate old data or something. And someone to do the work, of course. > It seems that this problem doesn't occur for .html files. The .html extension is hardcoded in an override list that can't be affected by either mimeTypes.rdf OR the operating system. See http://lxr.mozilla.org/seamonkey/source/uriloader/exthandler/nsExternalHelperAppService.cpp#376 I'd really rather not add things to that list unless we absolutely have to, frankly. But I suppose we could add SVG to it... But only if absolutely nothing else in the world uses the .svg extension. We've had bugs with the .css thing in there (because some spreadsheet format or something uses .css as an extension).

> Well, users need to be able to add them, yes. What are the ways in which a user can add an override? Is just choosing to save a file you're prompted to do something with one of them? I can't see how users would realise that's what they're doing. > I'd be fine with that changing, but we'd need to migrate old data or > something. And someone to do the work, of course. That sounds like a major task full of potential for profile corruption and the like. > I suppose we could add SVG to it... But only if absolutely nothing > else in the world uses the .svg extension. We've had bugs with the .css thing > in there (because some spreadsheet format or something uses .css as an > extension). I'd be interested to know what type of problems it has caused, but I'm not quite sure how to search for those bugs.

> What are the ways in which a user can add an override? In Seamonkey, you just open preferences and add one. In Firefox, I have no idea; the Firefox prefs UI doesn't really allow adding things, only editing ore removing existing ones. But then again Firefox UI is SEP as far as I'm concerned. > Is just choosing to save a file you're prompted to do something with one of > them? At the moment, yes. As I said, I'm fine with changing that. > I'd be interested to know what type of problems it has caused People's binary spreadsheet data being treated as text/css and hence rendering as plaintext in the browser window. And on the other hand, people's stylesheets being sent with the spreadsheet MIME type by Apache and getting ignored in standards mode (but that wasn't due to our override, of course, just due to the MIME type collision on the extension and Apache being dumb).

ajaxsketch absolutely ought to send its files with content-disposition: attachment rather than making up a mime type...

This is pretty brutal to leave lying in wait for users. Requesting 1.8.1. If someone can sketch a design approach here, I'll thank them relentlessly.

Comment 3?

I've made a start on a doc for end users that explains this bug and what they can do (of course not many will know what SVG is, let alone go to the effort of tracking down the source of this problem or fixing it). It also explains the problem slightly better than it's explained here I think. See http://developer.mozilla.org/en/docs/SVG:mimeTypes.rdf_corruption

Yeah, I was sort of looking for something a little more detail there. :) Anyway, thoughts on where it would be split, how we'd migrate data, etc. would be welcome as they come to anyone. Thanks!

So the only time the helper app dialog writes to mimeTypes.rdf is when it calls updateHelperAppPref() here: http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/toolkit/mozapps/downloads/src/nsHelperAppDlg.js.in&rev=1.33&mark=737,766,788,820,852#846 Unfortunately it _always_ creates an entry in mimeTypes.rdf, _regardless_ of the action you choose from the dialog (except if you cancel), see: http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/toolkit/mozapps/downloads/src/nsHelperAppDlg.js.in&rev=1.33&mark=737,766,788,820,852#766 I'm not sure if the bug is *only* in Firefox code, but this certainly seems wrong. If you choose the "Save to Disk" radio button and the "Do this automatically for files like this from now on" checkbox isn't checked, why would it write anything to mimeTypes.rdf? Having said that, it seems desirable to allow a record of some sort to be made if an application is chosen, even if the "Do this automatically..." checkbox isn't checked, so that the next time the dialog is brought up it shows the last app chosen. It's just unfortunate that file extensions can only be associated with one MIME type, and that the helper app dialog can make that the wrong MIME type.

How about we add SVG to the hardcoded override list (defaultMimeEntries) for Firefox 1.5.0.4 (or ASAP) and see if there are (m)any complaints? This is going to take time to come up with a good solution, and I would like to stop the escalating profile corruption going on out there until we can find one. I think the possibility of minor annoyance to a small number of people who use .svg to for non-SVG content on the Web is far outweighed by the damage that's being done because of this bug. To repeat what I said to bz in a private mail: There are an unknown and increasing numbers of people going around unable to view SVG with Mozilla because of this bug. SVG becomes less and less credible as this situation gets worse. People have enough hurdles to overcome to get their SVG working in moz, and if it becomes too common for SVG just not to work - seemingly randomly, for seemingly random users - devs won't feel they can use it. And it's not like the problem is going to go away as users updates moz. Since it's profile corruption, the problem just gets carried along and the number of people with this problem will on only increase, never decrease.

So bz and I have been having some peripheral chat about this off-list, but we've started to get back to discussion that should really be done in public, so with his permission I'm replying to his last email here. Boris Zbarsky wrote: > So let's back up. What problems are we trying to solve? [snip] The problem comes in three parts, I think: 1) When someone uses the helper app dialog to choose an action for a MIME type that moz doesn't recognize, an entry is added to the MIME service associating that MIME type with the file extension that the file happened to have. It doesn't matter if the action was to download it, or to open it with another app, or if the "remember this action" checkbox was checked or not. 2) The MIME service only allows a file extension to be associated with *one* MIME type (see the link below). Once a file extension has been stored in the MIME service, it seems from my experiments that as far as moz is concerned, that extension is forever tied to the MIME type it was associated with at the time it was added. [The problem with this is that in reality file extensions are often used for more than one type of file.] http://lxr.mozilla.org/seamonkey/source/netwerk/mime/public/nsIMIMEService.idl#58 3) It is quite common for SVG to be served incorrectly as image/svg, image/svg-xml, etc. Also the widely publicized ajaxSketch sends .svg files with the MIME type x-type/x-doc when you hit the save button to save your sketch (their way of forcing moz to prompt the user to save the file). Unfortunately, because of problems (1) and (2), once a user chooses to save or open one of these files using a helper app, the .svg file extension is forever associated with a MIME type that moz doesn't understand. Surprise, dear user! Now you can't even open the local SVG file you just saved _or_ view any embedded SVG on the Web. As for what sort of solution I'd like to see at this stage, I'd state the goals as this: A) Make sure that when our users choose to save or use a helper app to open one of these incorrectly served files, moz will not link the .svg file extension with the broken MIME type. B) After we've done that, figure out some way to *push* fixes to users who's profiles already associate the .svg file extension with a broken SVG MIME type so that they can once again open the SVG they saved locally and view embedded SVG on the Web. At this stage, I'm only concerned with fixing problems in the way Gecko stores MIME/file extension preferences or the way Firefox uses this storage in so far as it solves the problem of users not being able to open/view SVG. Of course it would be nice if the fixes also solved any problems that may exist in the way these prefs are stored, and the way Firefox uses them. I hope that answers your question.

> When someone uses the helper app dialog This is an app issue, not a core file handling issue, right? That is, changes to this behavior should be made by the UI folks, with possible back end changes to support whatever they want. > The MIME service only allows a file extension to be associated with *one* MIME > type Actually, it doesn't enforce this. But yes, generally speaking the idea is that given an extension we determine a type from it and handle that type. I'm not sure what can be changed here, given that Gecko's dispatch model is type-based. We could change that, I suppose... > The problem with this is that in reality file extensions are often used for > more than one type of file. Generally not on a single system, since all our tier 1 platforms sadly use extensions as their type identifier. :( Of course on an OS that used something else as a type identifier, the whole issue would not come up.... > It is quite common for SVG to be served incorrectly as image/svg, > image/svg-xml, etc OK. So here we hit the crux of the matter -- that your item #1 combined with this bustage on the part of other people causes grief. > _or_ view any embedded SVG on the Web. We're talking branch, not trunk, right? So as far as your goals go, here are some thoughts: A) How do we tell that a type is "broken"? I _think_ the reason we do the "save MIME info to mimeTypes.rdf" thing is that, generally speaking, it provides a better user experience than not remembering anything about the user's interaction with the file. Apparently that's not the case for SVG because of the many busted MIME types used. How can we detect that? Perhaps we should simply not store the URI extension unless it already maps to that type? Again, I'd love to hear feedback from Ben, Mike, and Neil. Another option we've considered is to split the MIME database into the extension-to-type mapping and the type-to-everything-else mapping, and have two separate levels of extension-to-type mapping. First level (the ones added automatically) is overridden by the OS and possibly our internal hint list, second level (the ones manually added by users) overrides the OS. Finally, we could add "svg" to our list of extensions that are forced to particular MIME types. At the moment we limit that list to startup-critical extensions as much as we can, and adding an extension to this list takes away all control over mapping that extension to a type from the user, so biesi and I are loath to add things to this list, generally. Especially because once added an extension really won't be able to be removed without causing bustages that never got fixed because the override wallpapered over them. B) Again, I don't see a way to determine what a "broken MIME type" is, in general. For any specific "SVG type", we can hook up a converter to convert to image/svg+xml, of course. We can even limit this to file channels, so that the ajaxSketch idiocy will continue to work. This converter would be a pretty small beastie; I can whip up a prototype for testing if you want. Note that a converter may not help with embedded SVG in Gecko 1.8, due to the inane way that works on the branch, but on trunk things will be happy. Again, there's the option of just adding the override, but again it makes me very unhappy. So I guess we should get some UI owner feedback on what they want the behavior to be, then sort out how to implement it.

Please excuse my ignorance of the dialog in question, but does anyone know why a) we always save a new MIME type b) we associate it with an extension?

We save the type because we want to present the user with whatever they selected as the preselected choice next time around. See http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/embedding/components/ui/helperAppDlg/nsHelperAppDlg.js&rev=1.70#569for the seamonkey code (and note the logic that sets the |needUpdate| boolean). The corresponding firefox code is at http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/toolkit/mozapps/downloads/src/nsHelperAppDlg.js.in&rev=1.33#737 Bug 86640 has various discussion that's relevant. As for why we save the extension... it's in the MIME info. The front end doesn't really know how it got into the MIME info, so it can't judge whether to leave it in there or not (and there might be multiple extensions too, of course). In this case, the way the extension ends up in the MIME info is that we pass the extension from the URI and the type from the channel into GetFromTypeAndExtension. That function calls GetMIMEInfoFromOS. What happens next is platform-dependent. On Unix, for example, if there is no default handler for the type, we get the info for the extension, set the passed-in type on it, and pass the whole thing out to the caller. If there is no info for the extension, we just create a new object containing the type and extension. I _think_ the idea was that associating extensions to types like this is a good heuristic for figuring out what extensions mean? That decision predates my time with this code; I could do some CVS archeology, but I'm not sure whether it would actually get us anywhere. So perhaps we should simply not use the URI extension (not pass it in to GetFromTypeAndExtension) if we have a type? biesi, do you recall whether that's actually fixing something?

(In reply to comment #14) > > The MIME service only allows a file extension to be associated with > > *one* MIME type > > Actually, it doesn't enforce this. Hmm, this was consistantly the case in my testing (either because of the front end or back end code, dunno yet), although the source comment says behavior is undefined. If I made an entry in mimeTypes.rdf associating the 'svg' extension with the correct SVG MIME type, then it prevented the extension from being associated with a bogus MIME type by any of the actions described above. > We're talking branch, not trunk, right? Yes, the breaking of embedded SVG on the Web only occurs on branch, since trunk has been fixed to use the MIME type the server sends for embedded content.

I notice that both bugs@bengoodger.com and beng@google.com are excluded from the list to whom bugzilla sends mail, despite being CC'ed on this bug. Since the front end code in questions is his, it would be nice to have an email address to CC that would actually get mail when we add comments.

So, really we need to handle mimetypes better, but its too late in the cycle to start figuring out a better solution. Should fix for Fx3 though. I think this is really a toolkit dlmgr bug, but plussing for 1.9a2 so we don't lose track of this.

Sounds like the real fix is to separate the two associations rather than recording both at the same time, as Boris said.

Not a regression from Fx2, not going to block on this. Dmose, do you have any input here?

Just to make sure... Mike, you meant to not mark this as wanted-1.9, right? Care to explain why?

Just voted this bug, hope this is the root of the problems which i was having. Firefox uploads .zip files with application/force-download mimetype and i'm checking it for the file to be a zip. Well i can easily fix it from my code with fileinfo magic or adding that mimetype.. But this bug wasn't easy to locate and i blamed almost everything else than firefox because i wasn't able to reproduce it. Fixing mimetypes from file uploads would be nice to me and all the other webcoders that don't think that some browsers would send file with totally bogus mimetype.

Hi, more than 1 year latter, it seems to me that this bug is still here. In fact when I try to upload a PDF file with firefox, the content-type detected is : "application/binary" instead of "application/pdf" When i take a look in mimeTypes.rdf, i have this two blocs of lines : <RDF:Description RDF:about="urn:mimetype:application/pdf" NC:fileExtensions="pdf" NC:description="Adobe Acrobat Document" NC:value="application/pdf" NC:editable="true"> <NC:handlerProp RDF:resource="urn:mimetype:handler:application/pdf"/> </RDF:Description> <RDF:Description RDF:about="urn:mimetype:application/binary" NC:fileExtensions="pdf" NC:description="Adobe Acrobat Document" NC:value="application/binary" NC:editable="true"> <NC:handlerProp RDF:resource="urn:mimetype:handler:application/binary"/> </RDF:Description> I'm using Firefox 3, from a new install (not from update).

Coming to this from bug 487976 (which is a Thunderbird bug), I'm wondering if there's a workaround for TB until this bug is fixed properly... Is there a way (perhaps by editing mimeTypes.rdf?) to force the OS's identification of the MIME type of a file to take precedence over what TB is currently deciding is the correct type? Or even, at a pinch, a way to force TB to ask the user to select the right MIME type? It seems like this bug isn't a high priority, and in the meantime I'd sure like to use a workaround if there is one, at least for TB. I'm particularly concerned about the case of files that have no extension, but which the OS can identify correctly.

We've run into a variant of this bug. It's in some core-level code, because the error is present in both Thunderbird (Mac and PC versions) and Firefox (the problem shows its head when using a webmail client to upload an attachment). Our specific version of the problem involves corrupted mimeTypes.rdf files. For reasons unknown (any clues would be very helpful), users' mimeTypes.rdf files will get corrupted and start marking file types as "application/applefile" instead of whatever they are supposed to be. We have seen this problem with PDFs and Word Documents. The "solution" is to simply delete mimeTypes.rdf, but doesn't solve the mystery of what caused the incorrect classification in the first place. Once mimeTypes.rdf is corrupted, it's not an issue of a file being sent from a broken mail client being unreadable on our side, it is now a problem of attachments being screwed up on the way out. This is a HUGE issue when sending to Exchange servers using Symantec anti-virus (at the server, not the client, level). When Symantec sees an application/applefile instead of what it should be seeing (application/pdf or application/msword), it sits quietly on the message and does not report an error to the sender or recipient. It will never deliver the message, but until the Exchange queue is purged (either manually or via reboot), the message will sit in the queue, unreported, and the sender will have no idea his message did not get through. At that point, the sender gets an error message explaining that the message timed out and was not delivered, but this could be weeks or months later. I know that part is not your problem, but it is a demonstration of why this issue is nontrivial. Symantec acknowledges the existence of the problem, but so far has not fixed it. I don't know what is causing the initial corruption of mimeTypes.rdf but the users who have experienced the problem made no attempt to change MIME types on purpose (I have had difficulty explaining the problem to them, in fact) and they are not power users. In some way, they must have accidentally caused the change, whether by accepting tainted messages or perhaps installing some application. The scary thing is that if they manage to break mimeTypes again, they won't notice and they will not have any indication of the problem until the mail server they sent the message to purges their queue. Again note that the problem has appeared on Macs and PCs, and in both Thunderbird and Firefox, so whatever is causing the problem, it must be easy to stumble upon.

I also ran into this problem recently with a SVG file wrongly being served as image/svg instead of image/svg+xml by a Bugzilla on freedesktop.org See my bug report there: http://bugs.freedesktop.org/show_bug.cgi?id=24192 It was extremely annoying, because my browser asked what to do for all the local SVG files after it happened.

Hi guys. Is there an Mozilla Developper to take this bug and work on it ? This bug still exists (since 2007).

Hi, is someone working on these bugs? This bug and the related bugs exists since 2005 or even loger. So the problem is over 5 Years old and still not fixed in recent versions. So i would guess this is the oldest open bug. This year I had several support requests, where these bugs where the root of the problem. Mostly pdf, but also doc and docx files. The number of duplicated,blocked or otherwise related bugs should have shown the importens of this bug. So please !!!! Fix it!!!

See https://bugzilla.mozilla.org/show_bug.cgi?id=373621#c45 for an explanation on why the wrong mime types exist.

This is a bug in Mozilla for not having a separate mimetypes.rdf clause to describe howto handle a given MIME-Type. It should have a Content-Disposition clause, rather than saying that it can wash its hands of interoperability by saying that it is a completely different type of data than the server said it is. "The server said it's application/pdf? Too bad, we say it's application/x-download, and we're going to force everything that we send it to to act as though it's only for downloading, never for processing as a PDF, just because of something that the user chose 10 years ago and because we never dreamed that this MIME-Type thing would actually be used for its intended purpose due to Microsoft's ham-handed first attempts at handling it that they've never really changed, either." Bug #373621 talks about a patch that exists solely for PDFs to ameliorate the effect of this bug, leaving absolutly every other filetype in the dark. The correct answer is to drop the Mozilla-specific disposition MIME-Types, start working with the correct ones, and convert all current mimetypes.rdf to use the new syntax in the updates' profile-compatibility checks. Or, if you're worried about people downgrading after an update (or breaking a standard which does not meet the needs of Mozilla's user base), then copy the information in mimetypes.rdf to another file, add the extra clause, and stop using mimetypes.rdf entirely. This is especially important if you expect people to use the information in their mimetypes.rdf files to mark other files that they might choose to publish -- because if they publish the MIME-Type as "application/x-download", it will cause confusion with other tools as well! This bug #332690 behavior has existed since before Firefox 1.5. c.f. https://developer.mozilla.org/en-US/docs/Web/SVG/mimeTypes.rdf_corruption The mimetypes.rdf behavior has been a fundamental error for literally as long as the Mozilla Foundation has had the inherited code from AOL (after it purchased Netscape Corporation). But, because it's such an ancient problem, it's a Sacred Cow which manifests in a LOT of bugs, many of which haven't been closed as duplicates of each other. And now that we've got web services that are actively using MIME-Types to alter how they handle uploaded files, the bug has matured and is starting to spread its seed over the web. It's vital that it is resolved correctly and conclusively, and long overdue given the proliferation of bugs that all have the same root cause of inappropriate reliance on incorrect alteration of mimetypes.rdf.

Should have been fixed in bug 306471.