Gecko allows third-parties to append data to the UA string by setting general.useragent.extra.* prefs. (Are there other ways?) This seems to be almost always junk that doesn't benefit the user in any way. It's also bad in the context of bug 572650. IE9 is taking similar action: http://blogs.msdn.com/b/ie/archive/2010/03/23/introducing-ie9-s-user-agent-string.aspx

To be specific, IE9 is standardizing the UA sent with HTTP requests but will still allow monkey-business with the UA accessible via navigator.userAgent.

This will catch most if not all Linux distributions by surprise, which like to add their build information to the UA string. Also, it is sometime necessary to spoof the UA string on purpose due to bad browser sniffing in web sites (affecting SeaMonkey and other non-Firefox Gecko browsers). So, like the other bugs in this category, there are both valid use cases and potential for abuse for the purpose of either advertisement or fingerprinting. Finding a compromise making the positive possible while avoiding the negative may be challenging...

(In reply to comment #2) > This will catch most if not all Linux distributions by surprise, which like to > add their build information to the UA string. What exactly do they add? > Also, it is sometime necessary > to spoof the UA string on purpose due to bad browser sniffing in web sites > (affecting SeaMonkey and other non-Firefox Gecko browsers). I suppose there would be something like general.useragent.appName and general.useragent.appVersion prefs. Firefox, SeaMonkey and Co. would define the defaults and users could change it in about:config. Third-parties could modify it too, but contrary to the currently supported way, they would have to actively decide on doing something malicious.

Camino uses this mechanism (general.useragent.extra.notfox) to append 'like Firefox/%FOX_APP_VERSION%' to the UA string. http://mxr.mozilla.org/camino/source/camino/resources/application/all-camino.js.in#73 For the obvious reason - too many sites break otherwise.

The obvious solution to this would be to support another pref like general.useragent.fxCompatToken which would probably default to false but could be enabled by Camino. The important part of this bug is that the generic way for appending all kinds of crap shouldn't be supported.

> (comment #3) What exactly do they add? I'm using openSUSE, which adds a SUSE/x.x.x-x.x build identifier, Fedora Core adds a Fedora/x.x.x-x.fcxx to identify its builds. Same with Ubuntu, adding number and code name as Ubuntu/x.xx (xxxx); and Debian, adding (Debian-x.x.x-x) at the end. I'd consider this a "valid advertisement" as they are actually customizing and repackaging the official releases. > I suppose there would be something like general.useragent.appName and > general.useragent.appVersion prefs. Firefox, SeaMonkey and Co. would define the > defaults and users could change it in about:config. If you limit it to a single preference again, the issue will be that spoofing the string will "hardwire" it and not reflect any updates. Right now, a SM user adding general.useragent.extra.firefox with "Firefox/3.5" will retain the official "SeaMonkey/2.0.6" part of the UA string, which gets updated without any further intervention. Having a single "SeaMonkey/2.0.6 Firefox/3.5" pref will need to be manually corrected with each update. > but contrary to the currently supported way, they would have to > actively decide on doing something malicious. I'm "confident" that the pseudo-applications you want to scare off will find a way to add their stuff to the UA string. Maybe a whitelist is the way to go?

> (comment #6) I'd consider this a "valid advertisement" as they are actually > customizing and repackaging the official releases. Obviously, it also make life easier in support forums and help desks to know the exact version and distribution used...

(In reply to comment #6) > I'm using openSUSE, which adds a SUSE/x.x.x-x.x build identifier, [...] > I'd consider this a "valid advertisement" as they are actually > customizing and repackaging the official releases. Do they customize the builds in any way that would be relevant to the Web? I don't think they do, nor should they. > If you limit it to a single preference again, the issue will be that spoofing > the string will "hardwire" it and not reflect any updates. Right now, a SM user > adding general.useragent.extra.firefox with "Firefox/3.5" will retain the > official "SeaMonkey/2.0.6" part of the UA string, which gets updated without > any further intervention. Having a single "SeaMonkey/2.0.6 Firefox/3.5" pref > will need to be manually corrected with each update. See comment 5.

I've added Wolfgang Rosenauer to the CC list who is providing the SUSE builds. (comment #5) Designing the general.useragent.fxCompatToken may certainly allow to trick buggy sites, designing it as a boolean pref using a fixed template would prevent arbitrary additions. It may be overly restrictive though and won't allow any vendor tokens to be added for custom builds.

If considered important we could probably support another magic pref for distributors (though not on Windows / OS X, I guess). I should also note that for the application token, a pref value like "SeaMonkey/2.0.6 Firefox/3.5" shouldn't actually be considered valid, exactly to prevent third-parties form appending random stuff.

Making it a configuration option at build time would also make it more tricky for those extension to just spoof it in the preferences, yet allowing a 3rd-party build to identify itself as such if desired. > "SeaMonkey/2.0.6 Firefox/3.5" Yes, you would likely need to sanitize general.useragent.app* to avoid someone making it "MEGASPOOF Firefox" + "3.5" or similar. Or just forgo the preferences altogether and only provide the build option along with the add-Firefox boolean preference. This would be a bit harsh again but limit the abuse mechanisms.

Sounds reasonable, we don't need to pref-ify everything. Some users might actually want to do things like replacing the official app name with some fantasy name, but we don't necessarily need to support this.

Actually the officially preferred way of spoofing for those SeaMonkey users that need it is to add a "NOT Firefox/x.y" part to the UA. We clearly are NOT Firefox, after all. In any case, any change of the default UA IMHO needs us to be able to apply a targeted site-specific spoofing mechanism similar to what Opera has, though I think we should have some way of telling the users that the site is broken and we need to (temporarily) lie to it to make it work. As long as we don't have such a mechanism, any larger UA change is doomed, IMHO, as it probably breaks large parts of the web.

(In reply to comment #13) > Actually the officially preferred way of spoofing for those SeaMonkey users > that need it is to add a "NOT Firefox/x.y" part to the UA. We clearly are NOT > Firefox, after all. "like" or "NOT" is an implementation detail, doesn't really matter. > In any case, any change of the default UA IMHO needs us to be able to apply a > targeted site-specific spoofing mechanism similar to what Opera has, though I > think we should have some way of telling the users that the site is broken and > we need to (temporarily) lie to it to make it work. As long as we don't have > such a mechanism, any larger UA change is doomed, IMHO, as it probably breaks > large parts of the web. For Firefox there would be no change from the default string, for other parties like SeaMonkey and Camino there might be a change in form of a locked down syntax with general.useragent.fxCompatToken set to true.

I don't have a strong opinion if I add SUSE or not to the useragent. It's convenient sometimes but I see the privacy/fingerprinting disadvantage. But then I think everyone needs to respect the final "spec". But please also note that sites are using that information for statistical purposes like some wordpress' statpress plugin and also wikipedia (as can be seen in this bugreport https://bugzilla.novell.com/show_bug.cgi?id=599328 ).

Note that banning UA altering shenanigans would not only reduce HTTP request bloat, improve privacy by reducing fingerprintability and not announcing every narcissistic addon you have installed (also potentially a security risk), but it would also help to stop potential breakage. Some of the addons that muck with the UA are idiots and break it in the process leaving the UA perpetually identifying itself as an old version of Firefox. See bug 577865 and its references.

I agree with comment #13 that having a more general way to show to misbehaving sites the UA string that it would like to see is certainly preferable. So, any solution here steering it by preferences has more of a workaround nature. (In reply to comment #14) > For Firefox there would be no change from the default string, for other parties > like SeaMonkey and Camino there might be a change in form of a locked down > syntax with general.useragent.fxCompatToken set to true. I'm sure we are missing some use case where people can't set the string exactly in the way that the web site expects by this mechanism (e.g., order of tokens), which was the reason for my somewhat careful phrasing of being harsh above. Anyway, it will be a trade-off but would work for me if implemented that way. (In reply to comment #15) > I don't have a strong opinion if I add SUSE or not to the useragent. > It's convenient sometimes but I see the privacy/fingerprinting disadvantage. I'd say provide the build-config option so that distributions can mark their builds, but then a general.useragent.hideVendor boolean pref (default: false) as a compromise to allow a user concerned about fingerprinting to hide it.

(In reply to comment #17) > I agree with comment #13 that having a more general way to show to misbehaving > sites the UA string that it would like to see is certainly preferable. So, any > solution here steering it by preferences has more of a workaround nature. > > (In reply to comment #14) > > For Firefox there would be no change from the default string, for other parties > > like SeaMonkey and Camino there might be a change in form of a locked down > > syntax with general.useragent.fxCompatToken set to true. > > I'm sure we are missing some use case where people can't set the string exactly > in the way that the web site expects by this mechanism (e.g., order of tokens), What tokens? It seems that the distribution thingy commonly comes before the app token, so we can preserve this. "like/not Firefox/foo" would come after the real app token, naturally. Other tokens wouldn't be allowed. However, also note that breaking a poorly written UA string parser is hardly a showstopper, as long as it's not considered a common problem.

I've recalled a case where a web site insisted on having the Firefox/x.x.x at the very end of the UA string, but that would be ensured by the proposed mechanism here. I agree that not any assumption a web site may make can be accommodated, the aim is to make sure that the user isn't confronted with a substantial portion of web sites being broken (no longer much of an issue here with those prefs, but the other related bugs may have that potential).

I think this discussion is odd at best - it's barking up the wrong tree. Security by obscurity won't work, especially with fingerprinting being around. As long as you don't kill *all* ways of modifying the UA (except at compile time), it's completely useless. Mind, though, that *not* being able to change your UA is a privacy issue in itself! Before the .extra. prefs were introduced, the only way to alter your UA were static changes to the .override pref, resulting in horribly out of sync UA, harming sites that deliver client-dependent content and support. The .extra. prefs were introduced to fix this. After all, you can delete those prefs... You could introduce a pref for "don't add .extra. stuff to my UA", but any addon which adds an .extra. might change that as well (unless it's locked).

(In reply to comment #20) > I think this discussion is odd at best - it's barking up the wrong tree. > Security by obscurity won't work, especially with fingerprinting being around. I'm not sure what you're talking about. This bug doesn't intent to introduce obscurity. It aims to support and lock down the cases we care about and not support others. > Before the .extra. prefs were introduced, the only way to alter your UA were > static changes to the .override pref, resulting in horribly out of sync UA, This bug isn't going to restore that behavior, either.

(In reply to comment #6) > I'm using openSUSE, which adds a SUSE/x.x.x-x.x build identifier, Fedora Core > adds a Fedora/x.x.x-x.fcxx to identify its builds. Same with Ubuntu, adding > number and code name as Ubuntu/x.xx (xxxx); and Debian, adding (Debian-x.x.x-x) FWIW, Debian stopped doing that 6 months ago. On the other hand, we are now adding "(like Firefox/x.y.z)", for the same reason Camino does (since we're shipping a rebranded browser).

(In reply to comment #21) > It aims to support and lock down the cases we care about and not > support others. Actually, you're trying to "fix" something that isn't broken! - Most UAs aren't altered anyway. - Not allowing to change the UA is a privacy issue in itself. - Restricting the way you can alter the UA just makes everyone's life harder without gaining anything.

(In reply to comment #23) > Actually, you're trying to "fix" something that isn't broken! > - Most UAs aren't altered anyway. Most means >50%? Maybe, although I doubt it. Even so, we want to protect the privacy of the notable rest. > - Not allowing to change the UA is a privacy issue in itself. How exactly? > - Restricting the way you can alter the UA just makes everyone's life harder > without gaining anything. Please tell me more. I assume "everyone" doesn't actually mean everyone? Otherwise, how does fit the idea that most UAs aren't altered anyway?

(In reply to comment #24) > (In reply to comment #23) > > Actually, you're trying to "fix" something that isn't broken! > > - Most UAs aren't altered anyway. > > Most means >50%? Maybe, although I doubt it. Instances of Firefox I found in a random excerpt of a log file: > Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 WebMoney Advisor > Mozilla/5.0 (Windows; U; Windows NT 5.0; ja; rv:1.9.2.7) Gecko/20100713 Firefox/3.6.7 GTB7.1 > Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.7) Gecko/20100713 Firefox/3.6.7 ( .NET CLR 3.5.30729; .NET4.0C) > Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.2.6) Gecko/20100625 Ant.com Toolbar 2.0 Firefox/3.6.6 > Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.2.7) Gecko/20100713 Firefox/3.6.7 > Mozilla/5.0 (Windows; U; Windows NT 6.0; ru; rv:1.9.1.11) Gecko/20100701 Firefox/3.5.11 (.NET CLR 3.5.30729) > Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4 AutoPager/0.6.1.22 > Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.7) Gecko/20100713 Firefox/3.6.7

(In reply to comment #24) > Most means >50%? Maybe, although I doubt it. This doesn't exactly sound as if you have _facts_ that there is an issue that needs fixing at all... :-P There still is just the *claim* that generally disallowing UA will be a step forward. Just because because $random_commercial_browser_vendor does $something doesn't automatically mean it's the Right Thing To Do. > Even so, we want to protect the privacy of the notable rest. You still fail to explain how this will enhance privacy in a reasonable manner. Either you can change the UA or you can't. If you still can, nothing changes in terms of privacy. If you can't, then it's a privacy issue in itself: > > - Not allowing to change the UA is a privacy issue in itself. > > How exactly? If I can't change the UA and I'm using a slightly uncommon version (like an OS/2 nightly build etc.), I can't do anything about it. Actually, why send an UA string at all? Shouldn't Mozilla/5.0 suit everyone, since even IE9 will be using it? :-P Isn't even Firefox/x.y.z an information a server shouldn't need to know? > > - Restricting the way you can alter the UA just makes everyone's life harder > > without gaining anything. > > Please tell me more. I assume "everyone" doesn't actually mean everyone? Oh, well, let's do semantics then, in absence of facts... *sigh* > Otherwise, how does fit the idea that most UAs aren't altered anyway? Of course, "everyone" means "everyone caring for the UA data", either because he needs help, or detect if he should/need/can provide a certain feature, etc. pp. Protecting privacy in a set of data that is *meant* to be contain meaningful information is hard, if not plain eyewash. If you really want to lock down the UA by making it as uniform as possible without hiding the different rendering engines, it should suffice to say "Gecko/2.0", everything else is just noise.

There's also those people who actually did heavily modify their UA because they wanted to, and because they want to be seen in stats (yes, I know, goes against the whole fingerprinting discussion, but some people just don't care). Those did use the .override to change the whole string, with the effect of not getting their UA updated for anything else any more, until the time when we introduced the .extra prefs and they could just register an addition instead of overriding everything.

(In reply to comment #26) > > Most means >50%? Maybe, although I doubt it. > > This doesn't exactly sound as if you have _facts_ that there is an issue that > needs fixing at all... :-P I thought it was common knowledge that junk gets added to UA strings quite often. However, for very rudimentary "facts", please see comment 25. (I could analyze the whole log file, and larger ones, but I don't really see the point since I have no doubt that it's a problem.) If you have opposing data, please provide it. > There still is just the *claim* that generally disallowing UA will be a step > forward. Just because because $random_commercial_browser_vendor does $something > doesn't automatically mean it's the Right Thing To Do. > > > Even so, we want to protect the privacy of the notable rest. > > You still fail to explain how this will enhance privacy in a reasonable manner. Again, I wrongly assumed this to be common knowledge. Let me ask you first, you do see the point of bug 572650, don't you? > > > - Not allowing to change the UA is a privacy issue in itself. > > > > How exactly? > > If I can't change the UA and I'm using a slightly uncommon version (like an > OS/2 nightly build etc.), I can't do anything about it. AFAIK we don't currently provide a way to hide the fact that you're running OS/2. > If you really want to lock down the UA by making it as uniform as possible > without hiding the different rendering engines, it should suffice to say > "Gecko/2.0", everything else is just noise. This would break the web, unfortunately.

(In reply to comment #27) > There's also those people who actually did heavily modify their UA because they > wanted to, and because they want to be seen in stats (yes, I know, goes against > the whole fingerprinting discussion, but some people just don't care). I don't think we care about this use case.

(In reply to comment #4) > Camino uses this mechanism (general.useragent.extra.notfox) to append 'like > Firefox/%FOX_APP_VERSION%' to the UA string. > http://mxr.mozilla.org/camino/source/camino/resources/application/all-camino.js.in#73 We also use general.useragent.extra.multilang in our Multilingual build to identify them for purposes of selecting the right download and right warning messages on our website. This is something that doesn't need to be exposed to the web at large, and I'd be happy to see it go if there were a private API I could use to get the same info to ensure the same good user experience on our website.

I dug through the code in order to work on a patch and discovered that general.useragent.override is actually still supported. We might want to keep that to give full power to extensions like User Agent Switcher. Third-parties such as those in comment 25 are unlikely to use that pref, I think.

This applies on top of the patch in bug 581783. kept: general.useragent.vendor, general.useragent.vendorSub (for Linux distributions, may want to revisit if we really need this) general.useragent.override (see comment 31) added: general.useragent.compatMode.firefox, general.useragent.compatMode.msie (convenient boolean prefs, so users and gecko-based apps don't need to wrestle with general.useragent.override) made read-only: nsIHttpProtocolHandler::vendor nsIHttpProtocolHandler::vendorSub nsIHttpProtocolHandler::product nsIHttpProtocolHandler::productSub nsIHttpProtocolHandler::misc removed: general.useragent.extra.* general.useragent.appName general.useragent.appVersion general.useragent.productComment general.useragent.vendorComment nsIHttpProtocolHandler::productComment nsIHttpProtocolHandler::vendorComment

Comment on attachment 460176 [details] [diff] [review] patch configure changes are fine, but you need to make the same changes in js/src.

(In reply to comment #32) > general.useragent.vendor, general.useragent.vendorSub (for Linux distributions, > may want to revisit if we really need this) Maybe compile the vendor string in instead of having a user writable pref and just have a bool pref to toggle showing it so there's nothing but the full override and friendly bool prefs left. > general.useragent.compatMode.firefox, general.useragent.compatMode.msie > (convenient boolean prefs, so users and gecko-based apps don't need to wrestle > with general.useragent.override) The Firefox compatibility flag is nice, but do we really need the IE one? A few years ago, maybe, but sites where you'd have to fully pretend to be IE are much more rare these days. A built-in flag to add "compatible; MSIE 8.0" doesn't feel needed, to me. (correct me if it's just me) The use case where I personally need to override on occasion is to pretend to be Windows because a site doesn't know what Linux is at all and won't work for no reason. A general.useragent.compatMode.windows to spoof the platform and OS to Windows would be nice, though I'm personally fine with the manual override.

(In reply to comment #34) > > general.useragent.vendor, general.useragent.vendorSub (for Linux distributions, > > may want to revisit if we really need this) > > Maybe compile the vendor string in instead of having a user writable pref and > just have a bool pref to toggle showing it so there's nothing but the full > override and friendly bool prefs left. This would be a good follow-up, I think, /if/ we want to support it at all. > The Firefox compatibility flag is nice, but do we really need the IE one? A few > years ago, maybe, but sites where you'd have to fully pretend to be IE are much > more rare these days. A built-in flag to add "compatible; MSIE 8.0" doesn't > feel needed, to me. (correct me if it's just me) This varies across countries, although cases where this would be needed are certainly becoming rarer.

(In reply to comment #32) > kept: ... > general.useragent.override (see comment 31) If you keep this, the removal of .extra. is just a waste of time, especially for others who will (need/want to) revert to changing that now; in fact, you're just returning to a past stage and resurrect its problems. Did you actually ever _read_ bug 274928?

It looks like there are references to many items on the patch removal list in that bug and why they were added in the first place.

This doesn't regress bug 274928. vendor and vendorSub are kept and Firefox doesn't use them. Besides, it seems clear that bug 274928 never took into account the abuse of .extra that we're seeing. (In reply to comment #37) > It looks like there are references to many items on the patch removal list in > that bug and why they were added in the first place. Can you be more explicit?

> (In reply to comment #37) > > It looks like there are references to many items on the patch removal list in > > that bug and why they were added in the first place. > > Can you be more explicit? Sure, it was a loose suggestion, so on further look over: I see that bug patch covers mainly adding the extra.* functionality based on some comments there (0-8, 18). I probably also read too much into comment.* suggestion there in # 5 (which I see must of been decided on as extra.*), so I guess I was falsely applying it to *.comment removal here and its not a match, nor in that bug at all. I also see it looks like your just changing appName and appVersion, rather than full removal? also I just meant it was noted these had been around before that for whatever reason prior. (In reply to comment #38) > This doesn't regress bug 274928. vendor and vendorSub are kept and Firefox > doesn't use them. I see this too (comment 0, 24), though I was reading the comments that added extra.* over vendor/vendorSub for Firefox to pref("general.useragent.extra.firefox", "Firefox/@APP_VERSION@"); Which I see is also different now anyway. One related question, maybe this is an obvious answer and implied already as gone completely, I'm assuming so, but a real life example for clarity: does removal of extra.* mean its entirely no longer usable at all for say extra.notfox = firefox/3.6.8 when a user uses Minefield and wants to add some spoofing compatibility? What then is the effect of changing extra.* and its relationship with the compatibility addition? Do you have some example UA's with this, that might help for further reference. > Besides, it seems clear that bug 274928 never took into > account the abuse of .extra that we're seeing. I agree with this (eg. .net clr)

Some of the earlier comments here seem to mention what the usage scenarios look like, but they are prior to the patch, so trying to get a handle on with this patch applied for some examples.

When this lands, a number of usages will just go back to using .override and lose any updating of the UA, but then, that looks like it's intended right now, as the paranoid people have taken over the UA string front in Mozilla, it seems, and not changing UAs seems fully in line with that.

(In reply to comment #41) > When this lands, a number of usages will just go back to using .override and > lose any updating of the UA, You're spreading FUD. What you should do instead is list use cases not currently covered, and the we can discuss whether we should and can enable them without .override.

(In reply to comment #39) > nor in that bug at all. I also see it looks like your just changing appName > and appVersion, rather than full removal? also I just meant it was noted these > had been around before that for whatever reason prior. appName and appVersion represent the Mozilla/5.0 token. We'd get rid of this if we could, but we can't due to compatibility constraints. There's no reason for this legacy token to be customizable. > One related question, maybe this is an obvious answer and implied already as > gone completely, I'm assuming so, but a real life example for clarity: does > removal of extra.* mean its entirely no longer usable at all for say > extra.notfox = firefox/3.6.8 when a user uses Minefield and wants to add some > spoofing compatibility? You'd use general.useragent.compatMode.firefox or general.useragent.override if the former doesn't suffice. For a Firefox nightly, compatMode.firefox would result in something like this: Mozilla/5.0 (X11; Linux i686; rv:2.0b3pre) Gecko/20100725 Minefield/4.0b3pre Firefox/4.0b3pre For a Gecko based browser it would look like this: Mozilla/5.0 (Windows; Windows NT 6.1; rv:1.9.1.10) Gecko/20100504 SeaMonkey/2.0.5 Firefox/3.5.10

(In reply to comment #43) > (In reply to comment #39) > > I also see it looks like your just changing appName > > and appVersion, rather than full removal? > > appName and appVersion represent the Mozilla/5.0 token. We'd get rid of this if > we could, but we can't due to compatibility constraints. Discussion for this is bug 527886. It's annoying to keep dragging around, but yeah, we're probably stuck with it for the foreseeable future. (In reply to comment #42) > (In reply to comment #41) > > When this lands, a number of usages will just go back to using .override and > > lose any updating of the UA, > > You're spreading FUD. What you should do instead is list use cases not > currently covered, and the we can discuss whether we should and can enable them > without .override. If any paranoia is warranted here then one suggestion may be to have .override expire at the end of the application session. Anyone who would want a permanent spoof would need to use one of the bool prefs or an extension to set the override on each application start. This would prevent broken UA modifications from stupid addons from taking a real hold via .override. (assuming other addons weren't malicious and constantly updated .override themselves)

Extensions abusing use of the UA string for what reason I don't understand except to advertise the extension to the world. There is an extension related specifically to the GetGlue website, which I see they might be tracking usage or making sure they can find the extension being installed, or making sure people have the extension to use the site, I though maybe an API can do this instead. I guess there is potential there for all addons who are abusing .extra.* to start always updating your .override UA instead, so your override UA keeps changing after installing abusive extensions. I think if this can be circumventable like Dave just said, that might be better and probably the subject of another UA bug.

(In reply to comment #44) [...] > If any paranoia is warranted here then one suggestion may be to have .override > expire at the end of the application session. Anyone who would want a permanent > spoof would need to use one of the bool prefs or an extension to set the > override on each application start. This would prevent broken UA modifications > from stupid addons from taking a real hold via .override. (assuming other > addons weren't malicious and constantly updated .override themselves) There is already an extension (User Agent Switcher) but even if there weren't, we could use user.js. The problem with _having_ to use general.useragent.override because partwise UA changes have gone away would be that we couldn't leave rv:2.0b2pre and SeaMonkey/2.1a3pre at their default (changing from version to version) while also adding something else to the UA, such as a Lightning version, maybe, or a Mnenhy version, or not just "Firefox/4.0b2pre" but "NOT Firefox/4.0b2pre" with NOT because SeaMonkey is not Firefox after all, or... Why not keep general.useragent.extra.* but have _them_ expire at the end of the session?

AMO ought to forbid add-ons setting .override or any other UA string modification without explicit user interaction (i.e. it's OK when the user wants to spoof it, but it shouldn't happen "just because").

(In reply to comment #45) > Extensions abusing use of the UA string for what reason I don't understand > except to advertise the extension to the world. [...] > I guess there is potential there for all addons who are abusing > .extra.* to start always updating your .override UA instead, so your override > UA keeps changing after installing abusive extensions. Many of these extensions are probably doing it only because we're providing the perfect tool to do it. We shouldn't provide this tool. Using the override pref for the same use case is more obviously malicious, so I doubt that many basically non-malicious vendors will do that. (In reply to comment #46) > There is already an extension (User Agent Switcher) but even if there weren't, > we could use user.js. The problem with _having_ to use > general.useragent.override because partwise UA changes have gone away would be > that we couldn't leave rv:2.0b2pre and SeaMonkey/2.1a3pre at their default > (changing from version to version) while also adding something else to the UA, > such as a Lightning version, maybe, or a Mnenhy version, or not just I don't see why this should be supported. > "Firefox/4.0b2pre" but "NOT Firefox/4.0b2pre" with NOT because SeaMonkey is not > Firefox after all, or... There's still SeaMonkey/foo, which makes it clear that it's not Firefox for anyone interested in this. As far as sites confusing Firefox with Gecko are concerned, SeaMonkey actually is Firefox... (In reply to comment #47) > AMO ought to forbid add-ons setting .override or any other UA string > modification without explicit user interaction (i.e. it's OK when the user > wants to spoof it, but it shouldn't happen "just because"). Yes.

(In reply to comment #46) > Why not keep general.useragent.extra.* but have _them_ expire at the end of the > session? See my comment 34. Support for appending is not enough. On more than one occasion I have had to use websites that require Windows or Mac OS X simply because they don't know the existence of anything else and won't let me proceed at all. I thus change the Linux OS declaration to Windows via .override. Some of these sites may or may not be able to be spoofed instead just by appending Windows and leaving Linux in, but that would be a weird UA listing two OSes and probably not expected to always work. One way or another, we need a way to spoof the whole thing or any particular part of it. (In reply to comment #47) > AMO ought to forbid add-ons setting .override or any other UA string > modification without explicit user interaction I mentioned this topic in a couple of these UA bugs, but yes, AMO is already planning on banning all non-critical UA modifications for all addons it hosts. The AMO addon validator was modified to report UA manipulation in bug 578725 (r71118).

Expiring override is a very excellent idea - .override will and must be available even for simple things like doing manual testing of how some web pages react to different UA strings. Also, targeted spoofing must still be able to work, and for the comment #49 use case it would also be a perfect match - but then, there seems to be tremendous opposition to even enabling such a crowdsourced, user-informing, per-site spoofing mechanism, let alone having it implemented, which I completely fail to understand. Interestingly, that one would allow us to even go further with UA changes, possibly as far as to remove that leading "Mozilla/5.0" things that is completely useless.

Using the full .override would be more self maintenance for all using nightlies I think who would want to use some backward version compatibility. I'm not personally concerned about using full override UAs because of testing nightlies, using bugzilla, etc is more important, but let me share this.. I guess I haven't validated trying to use extra.notfox = Firefox/4.0b3pre, since what we've seen already is sites only supporting official release numbers and or product/official release numbers strings, and they shouldn't have to allow nightlies to work, ie unofficial support. But this is the web, and that's not what I think Mozilla in general is looking for with open web efforts and currently using extra.notfox = Firefox/3.6.8 many times makes a site work. So alternatively having 1 more user pref to take care of the official release version compatibility would at least not affect the rest of the UA be affected for the other use cases where using the UA for bugzilla to file/report or verify bugs, find regressions is important too. I guess you could go as far as just compile in a official release version comp string.. and I realize build config efforts might not keep up with compile time. So it might make sense to have compMode.Firefox.version pref to be a single user pref and allow it to show only when the Firefox compatMode pref is flipped. I would think this compromise would well with easy maintenance with freedom to not have to worry about the .override for the cases mentioned. That I think would be real nice plus for users since it appears the patch looks to only cover the product name case, but using the same version of the current build, which could still break some sites as not compatible or not officially supported.

(In reply to comment #49) > (In reply to comment #46) > > Why not keep general.useragent.extra.* but have _them_ expire at the end of the > > session? > > See my comment 34. Support for appending is not enough. On more than one > occasion I have had to use websites that require Windows or Mac OS X simply > because they don't know the existence of anything else and won't let me proceed > at all. I thus change the Linux OS declaration to Windows via .override. Some > of these sites may or may not be able to be spoofed instead just by appending > Windows and leaving Linux in, but that would be a weird UA listing two OSes and > probably not expected to always work. One way or another, we need a way to > spoof the whole thing or any particular part of it. I didn't mean to remove support for .override. What I meant was (support .override with or without expire at end of session and) support .extra.* but expire at end of session rather than removing all support for them. Sorry if wasn't clear. Yes, your usecase make perfect sense if you need to visit some shtoop'd sites meant for Bill Gates's serfs only. (I guess you'll still have a problem if, like Korean banks, they require that your browser support ActiveX; but that's a different problem.)

Dennis: That sounds like something an extension like Nightly Tester Tools could make easier for you. We should be focused on end users here. Tony: Expiring .extra wouldn't do what we need this bug's solution to do. E.g. it wouldn't make a difference at all for extensions setting their pref in defaults.js or at startup.

Tony: Ah, I took your emphasis of _them_ to imply instead of rather than in addition to. Dão makes a good point in comment 53 though. Any route to appending arbitrary junk to the UA via a pref like this is a liability to this problem.

(In reply to comment #53) > Dennis: That sounds like something an extension like Nightly Tester Tools could > make easier for you. We should be focused on end users here. Yeah, I guess I tend to think end users need it easy too. With the specific case of using NTT, unfortunately, NTT only changes the titlebar text, not the actualy UA send over the web, and I don't see how to make it do otherwise which and doesn't really help the spoofing flexible case but I do understand the bugzilla use cases with it. Mossip didn't sound like he was wanting to focus on supporting/updating that tool anyway. :-/ I guess I'll have to test user agent switcher too. I also understand the liability issue that Dave notes in the previous comment with extra. cruft. Maybe the suggestion I had doesn't quite help this case.

I've filed bug 582421 for the idea to make .override expire.

(In reply to comment #55) > With the specific case of using NTT, unfortunately, NTT only changes the I know it doesn't do it, and I know it's currently unmaintained, but the point is that the task seems suitable and should be relatively easy for an extension /like/ Nightly Tester Tools.

(In reply to comment #57) > (In reply to comment #55) > > With the specific case of using NTT, unfortunately, NTT only changes the > > I know it doesn't do it, and I know it's currently unmaintained, but the point > is that the task seems suitable and should be relatively easy for an extension > /like/ Nightly Tester Tools. Gotcha.. Another side question related to appName/appVersion. Does the patch cross paths with bug 61071 or bug 115773 at all?

(In reply to comment #58) > Another side question related to appName/appVersion. Does the patch cross > paths with bug 61071 or bug 115773 at all? No.

Comment on attachment 460176 [details] [diff] [review] patch please see comment 32 for a summary of what this does

It'll be a few days at least before I can get to this.

I'm happy to sr, but I'd like to get dbaron's feedback since he added the .extra. feature and it solved a real problem with add-on useragent-wars we were having at the time.

If I recall, the problem it was solving was that, at the time, we didn't have a way to add both a token to indicate Firefox/Version and a token to indicate the Linux distro. I'm fine with saying that the Linux distros can't add to the UA string, and likewise that nobody else can either. My one worry is that extensions might use general.useragent.override to add something to it in ways that break when they update versions, or that break across platforms, etc. I'm not sure that such a thing would feel "obviously malicious" (comment 48) to the person doing it, especially if they've been instructed by their boss to add a token to the UA string and given X days to implement the feature. But I think overall this is probably a good thing.

Comment on attachment 460176 [details] [diff] [review] patch >+++ b/layout/tools/reftest/reftest.js > for each (var prop in [ "userAgent", "appName", "appVersion", >- "vendor", "vendorSub", "vendorComment", >- "product", "productSub", "productComment", >+ "vendor", "vendorSub", >+ "product", "productSub", > "platform", "oscpu", "language", "misc" ]) > sandbox.http[prop] = hh[prop]; Shouldn't there be a test that verifies that the old properties do not exist (somewhere, maybe not this one)? >+++ b/modules/libpref/src/init/all.js >+pref("general.useragent.compatMode.firefox", false); >+pref("general.useragent.compatMode.msie", false); I'm not enamored of the MSIE compat idea. I buy the argument for Firefox-compat for non-Firefox Gecko apps, but it's something that will be decided on a per-application basis and not something users will flip. MSIE compat is a user-facing feature--no Gecko-based app is going to select that by default--but users who need it are probably better served by complete spoofing through add-ons like user-agent switcher. > mMisc.AssignLiteral("rv:" MOZILLA_VERSION); >+ mApplication.AssignLiteral(MOZ_APP_DISPLAYNAME "/" MOZ_APP_VERSION); Shouldn't this use the APP_UA_NAME from the extra pref you removed? A little problematic since that was only defined in a /browser makefile, but I assume the extra processing was there for good reason. Actually I'd prefer a completely separately defined UA_NAME so that nightly builds are always "Firefox" in the UA despite the display name being Minefield in the user interface. I guess that can be handled in the other bug that covers that topic, but seems a shame not to do both at the same time. On the other hand maybe your intention is that we just flip the compat pref for nightlies so it says "Minefield/x.y Firefox/x.y" but that seems kind of lame. >+ mUserAgent += ' '; >+ mUserAgent += mApplication; >+ >+ if (!mCompatFirefox.IsEmpty()) { >+ mUserAgent += ' '; >+ mUserAgent += mCompatFirefox; >+ } My personal preference would be to do it the way Camino has, as a comment rather than a product token, and I prefer "(like Firefox/x.y)" over "(not Firefox/x.y)". But I wouldn't deny an sr over that. Waiting for an actual review and feedback from dbaron (who may remember some .extra. edgecase we were trying to solve) before formally stamping sr+

(In reply to comment #63) > My one worry is that extensions might use general.useragent.override to add > something to it in ways that break when they update versions Yes, that happened all the time -- the UA would freeze at the version the user was on when the add-on set the override pref. I guess we'll hope AMO code reviews and social pressure prevent user-agent wars in the future. It will help that Microsoft is backing the same principles.

(In reply to comment #65) > (In reply to comment #63) > > My one worry is that extensions might use general.useragent.override to add > > something to it in ways that break when they update versions > > Yes, that happened all the time -- the UA would freeze at the version the user > was on when the add-on set the override pref. This is what will happen again. Those who felt like touching the UA before still will feel so, but now resort to changing the .override again. This bug here just *harms* the user experience without gaining anything.

(In reply to comment #65) > (In reply to comment #63) > > My one worry is that extensions might use general.useragent.override to add > > something to it in ways that break when they update versions > > Yes, that happened all the time -- the UA would freeze at the version the user > was on when the add-on set the override pref. > > I guess we'll hope AMO code reviews and social pressure prevent user-agent wars > in the future. It will help that Microsoft is backing the same principles. Also, bug 582421 proposes a way to fix that.

(In reply to comment #64) > >+++ b/layout/tools/reftest/reftest.js > > for each (var prop in [ "userAgent", "appName", "appVersion", > >- "vendor", "vendorSub", "vendorComment", > >- "product", "productSub", "productComment", > >+ "vendor", "vendorSub", > >+ "product", "productSub", > > "platform", "oscpu", "language", "misc" ]) > > sandbox.http[prop] = hh[prop]; > > Shouldn't there be a test that verifies that the old properties do not exist > (somewhere, maybe not this one)? Umm, I don't know, adding a (regression?) test for the lack of support for certain properties seems a little strange to me. > >+++ b/modules/libpref/src/init/all.js > >+pref("general.useragent.compatMode.firefox", false); > >+pref("general.useragent.compatMode.msie", false); > > I'm not enamored of the MSIE compat idea. I buy the argument for Firefox-compat > for non-Firefox Gecko apps, but it's something that will be decided on a > per-application basis and not something users will flip. MSIE compat is a > user-facing feature--no Gecko-based app is going to select that by default--but > users who need it are probably better served by complete spoofing through > add-ons like user-agent switcher. User Agent Switcher might not be available for arbitrary Gecko apps. Supporting the pref seemed cheap enough, but I can remove it if you think it won't ever be handy. > > mMisc.AssignLiteral("rv:" MOZILLA_VERSION); > >+ mApplication.AssignLiteral(MOZ_APP_DISPLAYNAME "/" MOZ_APP_VERSION); > > Shouldn't this use the APP_UA_NAME from the extra pref you removed? A little > problematic since that was only defined in a /browser makefile, but I assume > the extra processing was there for good reason. > > Actually I'd prefer a completely separately defined UA_NAME [snip] I'll look into this. > so that nightly > builds are always "Firefox" in the UA despite the display name being Minefield > in the user interface. I guess that can be handled in the other bug that covers > that topic, but seems a shame not to do both at the same time. I really don't want to cover this in this bug. > >+ mUserAgent += ' '; > >+ mUserAgent += mApplication; > >+ > >+ if (!mCompatFirefox.IsEmpty()) { > >+ mUserAgent += ' '; > >+ mUserAgent += mCompatFirefox; > >+ } > > My personal preference would be to do it the way Camino has, as a comment > rather than a product token, and I prefer "(like Firefox/x.y)" over "(not > Firefox/x.y)". But I wouldn't deny an sr over that. I don't really have an opinion about this, I can change it.

(In reply to comment #64) > I'm not enamored of the MSIE compat idea. I buy the argument for Firefox-compat > for non-Firefox Gecko apps, but it's something that will be decided on a > per-application basis and not something users will flip I agree that the MSIE thing is probably not that good, as sites expecting MSIE expect a differently formed UA string. We currently have _users_ adding the .extra.notfirefox="NOT Firefox/4.0" token and I fully expect users to set the new one via about:config. Actually, I'm thinking about adding a UI pref for it in SeaMonkey.

And BTW, doing the FF compat token as a comment probably would break a number of sites as well, I'd prefer to do it right and the way Firefox does it as well, i.e. as a product token. But it always should be off by default - though that's of course an app choice.

While I agree that extensions should leave the UA string alone, I don't think you can achieve that with removing .extra.*, because there's still .override, and we need that pref for users. Bug 582421 therefore proposes to make that pref temporary and reset it on every start, but I think that's an ill effect for users and won't help with rough extensions. See bug 582421 comment 17. So, while I agree with the goal of this bug, I don't think you'll achieve it and rather have ill effects.

Added MOZ_APP_UA_NAME, removed general.useragent.compatMode.msie

So can someone describe what the current result of the patch is (does it still match comment 32?) and its impact on various currently existing Gecko/Firefox UA strings (e.g. for non-Firefox apps, Linux distributions, etc)?

It matches comment 32 except for the updates in comment 72. All in all: kept: general.useragent.vendor general.useragent.vendorSub general.useragent.override added: general.useragent.compatMode.firefox MOZ_APP_UA_NAME (if undefined, this is derived from MOZ_APP_DISPLAYNAME) made read-only: nsIHttpProtocolHandler::vendor nsIHttpProtocolHandler::vendorSub nsIHttpProtocolHandler::product nsIHttpProtocolHandler::productSub nsIHttpProtocolHandler::misc removed: general.useragent.extra.* general.useragent.appName general.useragent.appVersion general.useragent.productComment general.useragent.vendorComment nsIHttpProtocolHandler::productComment nsIHttpProtocolHandler::vendorComment None of this effects the stock Firefox UA string, except on Ubuntu, which is the only user of vendorComment that I know of. " (lucid)" would be dropped from "Ubuntu/10.04 (lucid)". It shouldn't make a difference for most other Gecko apps either. It's going to make a difference a difference for non-Firefox Gecko apps claiming to be Firefox. general.useragent.compatMode.firefox was added for this case, but that leads to a fixed format, i.e. it doesn't preserve various formats currently being used, such as "(like Firefox/3.6)".

So, general.useragent.appName is removed in favor of a compile-time define. App version is now derived from <src>/config/version.txt , i.e. also compile-time define? > It shouldn't make a difference for most other Gecko apps either. What about embedded Gecko in third party apps? What about XULRunner apps? (They'll want to say e.g. "tthome/2.0" as appname, but many will not want to compile themselves.) I think that'll be a problem for them, assuming they care about their UA string.

(In reply to comment #76) > So, general.useragent.appName is removed in favor of a compile-time define. > App version is now derived from <src>/config/version.txt , i.e. also > compile-time define? No, general.userAgent.extra.foo is removed in favor of the compile-time defines. > > It shouldn't make a difference for most other Gecko apps either. > > What about embedded Gecko in third party apps? Mozilla/2.0, I think. > What about XULRunner apps? (They'll want to say e.g. "tthome/2.0" as appname, > but many will not want to compile themselves.) > I think that'll be a problem for them, assuming they care about their UA > string. XULRunner/2.0. They might care about being in the UA string without compiling, but then I'm not sure they have good enough reasons... It might be used for communicating with TomTom services or something, but they control these requests, i.e. they can use other means than the UA string. By the way, my patch doesn't remove general.userAgent.vendor and general.userAgent.vendorSub, and it might make sense for TomTom to set this. I think we should look into getting rid of these prefs as a follow-up, though.

Haven't taken a look at the patch, but what about firefox-on-top-of-xulrunner ?

(In reply to comment #78) > Haven't taken a look at the patch, but what about firefox-on-top-of-xulrunner ? I guess you would set general.useragent.compatMode.firefox = true to add the Firefox/version token.

Okay, I took a look at the patch, and I think you should just use the application name from nsIXULAppInfo.

> I guess you would set general.useragent.compatMode.firefox = true to add the > Firefox/version token. ... which would be wrong for anything that is not firefox but running on top of xulrunner. Think iceweasel, icecat, or, much more non-firefox, conkeror.

I'll look into using nsIXULAppInfo.

nsIXULAppInfo fallback added for the app name

App name in application.ini (nsIXULAppInfo) can very well and likely have spaces, but they are not allowed in the app name (they would form a different application token). Similary, other characters are not allowed either. Please see the spec and transform the XUL app name into a canonical UA string app name before using it.

Comment on attachment 464414 [details] [diff] [review] patch v3 >+ nsCOMPtr<nsIXULAppInfo> appInfo = >+ do_GetService("@mozilla.org/xre/app-info;1"); >+ >+ mAppName.AssignLiteral(MOZ_APP_UA_NAME); >+ if (mAppName.Length() == 0 && appInfo) { >+ appInfo->GetName(mAppName); >+ appInfo->GetVersion(mAppVersion); >+ } else { >+ mAppVersion.AssignLiteral(MOZ_APP_VERSION); >+ } I'm wondering if that shouldn't be the contrary. i.e. MOZ_APP_UA_NAME being the fallback. Or even, if it is necessary at all. Are there still embedders not providing appinfo, or that would like the UA to include something different from it ?

Yes, ability to override is good, I may want something else in UA string than I have as xulrunner app name (which is displayed in UI etc.).

(In reply to comment #86) > Yes, ability to override is good, I may want something else in UA string than I > have as xulrunner app name (which is displayed in UI etc.). Except MOZ_APP_UA_NAME is hard coded at build time, so if your application is a xulrunner app, it won't be using whatever it likes. Which means the solution would probably be to make it part of nsIXULAppInfo. It could be initialized to some filtered value of the application name if not given in application.ini or equivalent.

(In reply to comment #84) > App name in application.ini (nsIXULAppInfo) can very well and likely have > spaces, but they are not allowed in the app name (they would form a different > application token). Similary, other characters are not allowed either. Please > see the spec and transform the XUL app name into a canonical UA string app name > before using it. So, US-ASCII 33-126 except ()<>@,;:\"/[]?={}. Is there a convenient API to sanitize the string? I suppose StripChars(" ()<>@,;:\\\"/[]?={}") isn't sufficient. (In reply to comment #87) > > Yes, ability to override is good, I may want something else in UA string than I > > have as xulrunner app name (which is displayed in UI etc.). > > Except MOZ_APP_UA_NAME is hard coded at build time, so if your application is a > xulrunner app, it won't be using whatever it likes. XULRunner doesn't set MOZ_APP_UA_NAME.

So I'm reading Microsoft's documentation here: http://blogs.msdn.com/b/ie/archive/2010/03/23/introducing-ie9-s-user-agent-string.aspx And the way I'm reading it is: "We'll send the short header for any HTTP requests, but you can still access the full user agent through Javascript using navigator.userAgent" But this bug is removing everything, right?

> > Except MOZ_APP_UA_NAME is hard coded at build time, so if your application is a > > xulrunner app, it won't be using whatever it likes. > > XULRunner doesn't set MOZ_APP_UA_NAME. The point is, xulrunner app will be limited to using the filtered application name, if you keep it the current way. I personally don't care, but apparently, BenB does.

[xulrunner] > Except MOZ_APP_UA_NAME is hard coded at build time Yes, that's exactly the problem that this bug introduces.

(In reply to comment #89) Yes, as stated in comment 1, IE9 isn't applying their lockout to the JS accessible UA string but the idea here is for us to ban amending entirely. It causes too many problems and there's no real reason to continue to support it. If one route were left open it would also continue to encourage bad practices. Having two potentiality different UAs also doesn't seem like the best idea.

I'm not going to be able to get to this before I go on vacation, unfortunately.

Comment on attachment 464414 [details] [diff] [review] patch v3 >diff --git a/js/src/configure.in b/js/src/configure.in >--- a/js/src/configure.in >+++ b/js/src/configure.in >@@ -5035,17 +5035,22 @@ AC_SUBST(MOZ_OS2_USE_DECLSPEC) > AC_SUBST(MOZ_POST_DSO_LIB_COMMAND) > AC_SUBST(MOZ_POST_PROGRAM_COMMAND) > AC_SUBST(MOZ_TIMELINE) > AC_SUBST(WINCE) > AC_SUBST(WINCE_WINDOWS_MOBILE) > > AC_SUBST(MOZ_APP_NAME) > AC_SUBST(MOZ_APP_DISPLAYNAME) >+AC_DEFINE_UNQUOTED(MOZ_APP_UA_NAME, "$MOZ_APP_UA_NAME") >+AC_SUBST(MOZ_APP_UA_NAME) >+AC_DEFINE_UNQUOTED(MOZ_APP_VERSION, "$MOZ_APP_VERSION") > AC_SUBST(MOZ_APP_VERSION) >+AC_DEFINE_UNQUOTED(FIREFOX_VERSION, "$FIREFOX_VERSION") >+AC_SUBST(FIREFOX_VERSION) > > AC_SUBST(MOZ_PKG_SPECIAL) > > AC_SUBST(MOZILLA_OFFICIAL) > > dnl win32 options > AC_SUBST(MOZ_MAPINFO) > AC_SUBST(MOZ_BROWSE_INFO) I think this is unnecessary. Why would js/src need this? (We don't try to keep the configure.in's in sync) Other than that the build bits look good.

(In reply to comment #66) > This is what will happen again. Those who felt like touching the UA before > still will feel so, but now resort to changing the .override again. This bug > here just *harms* the user experience without gaining anything. Keep in mind that even without .override or .override being just a temporal setting there still would be ways for changing the user-agent string secretly and without any benefit for the user. Therefore, my suggestion is: 0.) Leave .override as it is. 1.) Enact fair-play rules for Fx extensions (like the AMO rules but meant for all extensions including those bundled with software) which make clear that this kind of behaviour is frowned upon. 2.) Put offending add-ons on the blacklist.

Dao, do you think this should block 4? Since we're doing a bunch of UA changes, it makes sense to me that this should be done at approximately the same time, so we can message to the various interested parties that they Should Not Do This Anymore. If you agree, nom for blocking?

I think we need to fix this asap. Cases like those in comment 25 are very common on Windows at least, adding fingerprinting surface and overhead to each and every HTTP request. IE is the only other browser with a similar problem, which MS is fixing in IE9.

Comment on attachment 464414 [details] [diff] [review] patch v3 Comments on the patch: (apologies if these have already been raised and resolved; I just skimmed the bug.) I've accomplished the UA_NAME bits in your patch in bug 588874, so all the various browsers that want to be called Firefox can do so. If you'd like, you can use my work there and rebase your patch. I'd recommend making "compatMode.firefox" a purely build-time thing. Fennec will always want it; I'm not sure about SeaMonkey (KaiRo?). I don't have a strong opinion though. I suppose I can see value for people wanting to flip it, especially if something like SeaMonkey doesn't want it on by default... but I don't see a case for wanting to turn it off if the product sets it on by default. I'd also recommend making vendor/vendorSub specified at build time (VENDOR_UANAME and VENDOR_VERSION?). Is there a good reason for people to override it at runtime? If we're trying to support distros, build time should be fine. I think the order of tokens should be "[Firefox/version] productName/productSub [vendor/vendorSub]". This would remain consistent with what Fennec currently does, and makes sense to me from an order-of-importance standpoint. (If you want to pretend to be Firefox-compatible, that should be the first thing we advertise.) What do you think?

(In reply to comment #98) > I'd recommend making "compatMode.firefox" a purely build-time thing. Fennec > will always want it; I'm not sure about SeaMonkey (KaiRo?). I don't have a > strong opinion though. I suppose I can see value for people wanting to flip it, > especially if something like SeaMonkey doesn't want it on by default... but I > don't see a case for wanting to turn it off if the product sets it on by > default. > > I'd also recommend making vendor/vendorSub specified at build time > (VENDOR_UANAME and VENDOR_VERSION?). Is there a good reason for people to > override it at runtime? If we're trying to support distros, build time should > be fine. A typical use case for both: xulrunner applications such as conkeror will want to set their app and vendor name/sub at runtime (since build time, here, would be set in stone in xulrunner)

(In reply to comment #99) > A typical use case for both: xulrunner applications such as conkeror will want > to set their app and vendor name/sub at runtime (since build time, here, would > be set in stone in xulrunner) Likewise for embedding applications like kazehakase or galeon.

Some Linux distros, e.g. Debian, run Firefox (eh, Iceweasel) and Thunderbird (Iceowl) from a shared xulrunner binary.

(In reply to comment #98) > I've accomplished the UA_NAME bits in your patch in bug 588874, so all the > various browsers that want to be called Firefox can do so. If you'd like, you > can use my work there and rebase your patch. It would actually make more sense for bug 588874 to depend on the patch here, since it would just mean removing the lines added here: diff --git a/browser/branding/nightly/configure.sh b/browser/branding/nightly/configure.sh --- a/browser/branding/nightly/configure.sh +++ b/browser/branding/nightly/configure.sh @@ -1,1 +1,2 @@ MOZ_APP_DISPLAYNAME=Minefield +MOZ_APP_UA_NAME=Minefield diff --git a/browser/branding/unofficial/configure.sh b/browser/branding/unofficial/configure.sh --- a/browser/branding/unofficial/configure.sh +++ b/browser/branding/unofficial/configure.sh @@ -1,1 +1,2 @@ MOZ_APP_DISPLAYNAME="MozillaDeveloperPreview" +MOZ_APP_UA_NAME="MozillaDeveloperPreview" > I suppose I can see value for people wanting to flip it, > especially if something like SeaMonkey doesn't want it on by default... but I > don't see a case for wanting to turn it off if the product sets it on by > default. Supporting users to opt in seems important while preventing users from opting out seems like a non-priority, so I'd rather leave this as is. > I'd also recommend making vendor/vendorSub specified at build time > (VENDOR_UANAME and VENDOR_VERSION?). Is there a good reason for people to > override it at runtime? If we're trying to support distros, build time should > be fine. I'm reluctant to widening this bug's scope... I think we should get rid of the vendor stuff altogether, though in a separate bug. If we do this, communicating the build time change to distros would be a wasted effort. If we're concerned that add-ons start setting the vendor prefs while they are still supported, we could easily mitigate this by making them unix-only. > I think the order of tokens should be "[Firefox/version] productName/productSub > [vendor/vendorSub]". This would remain consistent with what Fennec currently > does, and makes sense to me from an order-of-importance standpoint. (If you > want to pretend to be Firefox-compatible, that should be the first thing we > advertise.) Fine by me, I don't really have an opinion on this.

(In reply to comment #98) > I think the order of tokens should be "[Firefox/version] productName/productSub > [vendor/vendorSub]". Sorry, I meant: product/productSub [Firefox/version] appName/appVersion [vendor/vendorSub] (In reply to comment #102) > It would actually make more sense for bug 588874 to depend on the patch here, > since it would just mean removing the lines added here: Yeah, you're right -- you're doing something slightly different with your flag, since you pull it from nsIXULAppInfo if it's unset. I'll update my patches over there. I'm going to be blogging about UA changes for Fx4 today. I think I'll mention that we're changing the prefs that control them, but leave the details for a later post, in case things change here. Does that sound OK?

(In reply to comment #103) > I'm going to be blogging about UA changes for Fx4 today. I think I'll mention > that we're changing the prefs that control them, but leave the details for a > later post, in case things change here. Does that sound OK? Yep.

(In reply to comment #98) > I'd recommend making "compatMode.firefox" a purely build-time thing. Ok for leaving the default for that as a build option, but why refusing to the user to change it? I don't see any harm done keeping the preference, if Fennec wants it on and SeaMonkey wants it off by default for their good reasons, the user nevertheless should be able to change that "factory setting" if he or she runs into issues with a specific site. Any fingerprinting impact of a boolean pref/token should be minimal.

(In reply to comment #105) > Any fingerprinting impact of a boolean pref/token should be minimal. That's not really true. If only a very small portion of users toggle it to a non-default setting then those few users will be highly fingerprintable when combined with other things like the accept language(s) and OS/platform. Though, non-Firefox users are already really highly fingerprintable due to their relatively small population size, so I don't think it's worth worrying about more. Regardless, I do agree that it might be a good idea to leave the pref available.

Comment on attachment 464414 [details] [diff] [review] patch v3 Talked to Johnny, and he said he could take the review, so I'm gonna switch it over. We have a couple days to land this and bz's way overworked ;)

This reorders the tokens as suggested in comment 103. (On top of patch v3.)

Dao, if you'd like I can land this and the other UA string patches in one go, once this gets sr.

Comment on attachment 464414 [details] [diff] [review] patch v3 >-APP_UA_NAME = $(shell echo $(MOZ_APP_DISPLAYNAME) | sed -e's/[^A-Za-z]//g') Equivalent logic is now missing. For apps that defile MOZ_APP_UA_NAME they can manually make sure they meet this requirement (a comment to that effect in the branding file would be nice) but what about the XULRunner case where you pick up the name from nsIXULAppInfo? >+++ b/browser/branding/nightly/configure.sh > MOZ_APP_DISPLAYNAME=Minefield >+MOZ_APP_UA_NAME=Minefield >+++ b/browser/branding/unofficial/configure.sh > MOZ_APP_DISPLAYNAME="MozillaDeveloperPreview" >+MOZ_APP_UA_NAME="MozillaDeveloperPreview" Thunderbird and SeaMonkey are going to need the equivalent when this lands, right? I don't expect you to check into comm-central, but please two bugs (one each) and nominate for their appropriate blocking flags so they catch the new requirement. Don't we have an "official" branding file that will need to be updated, too? >+ nsCOMPtr<nsIXULAppInfo> appInfo = >+ do_GetService("@mozilla.org/xre/app-info;1"); >+ >+ mAppName.AssignLiteral(MOZ_APP_UA_NAME); >+ if (mAppName.Length() == 0 && appInfo) { >+ appInfo->GetName(mAppName); >+ appInfo->GetVersion(mAppVersion); >+ } else { >+ mAppVersion.AssignLiteral(MOZ_APP_VERSION); >+ } Here's where we need to filter unwanted characters out of mAppName. It's OK to be stricter than the spec, but not looser. Allow A-Za-z0-9 plus hyphen and underscore. >+ mUserAgent += ' '; >+ mUserAgent += mProduct; >+ mUserAgent += '/'; >+ mUserAgent += mProductSub; There's another bug about dropping the builddate. Should the last two lines above be conditional on a non-empty mProductSub, or do you want to just leave this as-is and let the other bug zap it?

(In reply to comment #110) > >+++ b/browser/branding/nightly/configure.sh > > MOZ_APP_DISPLAYNAME=Minefield > >+MOZ_APP_UA_NAME=Minefield > >+++ b/browser/branding/unofficial/configure.sh > > MOZ_APP_DISPLAYNAME="MozillaDeveloperPreview" > >+MOZ_APP_UA_NAME="MozillaDeveloperPreview" > > Thunderbird and SeaMonkey are going to need the equivalent when this lands, > right? I don't expect you to check into comm-central, but please two bugs (one > each) and nominate for their appropriate blocking flags so they catch the new > requirement. They shouldn't have to do anything, they'd get "Thunderbird" and "SeaMonkey" automatically. > Don't we have an "official" branding file that will need to be updated, too? "Firefox" will be used automatically here as well. > >+ nsCOMPtr<nsIXULAppInfo> appInfo = > >+ do_GetService("@mozilla.org/xre/app-info;1"); > >+ > >+ mAppName.AssignLiteral(MOZ_APP_UA_NAME); > >+ if (mAppName.Length() == 0 && appInfo) { > >+ appInfo->GetName(mAppName); > >+ appInfo->GetVersion(mAppVersion); > >+ } else { > >+ mAppVersion.AssignLiteral(MOZ_APP_VERSION); > >+ } > > Here's where we need to filter unwanted characters out of mAppName. It's OK to > be stricter than the spec, but not looser. Allow A-Za-z0-9 plus hyphen and > underscore. Is there an API similar to StripChars but with a character whitelist rather than a blacklist? > >+ mUserAgent += ' '; > >+ mUserAgent += mProduct; > >+ mUserAgent += '/'; > >+ mUserAgent += mProductSub; > > There's another bug about dropping the builddate. Should the last two lines > above be conditional on a non-empty mProductSub, or do you want to just leave > this as-is and let the other bug zap it? I'll let the other bug deal with this.

(In reply to comment #111) > Is there an API similar to StripChars but with a character whitelist rather > than a blacklist? I don't think so; you gotta make a little inline islegal(char) function and loop over the string. Should be a few LOC.

Comment on attachment 464414 [details] [diff] [review] patch v3 sr=dveditz I filed bug 591125 on the name sanitization. It's not a problem for the homegrown apps that I checked in the repo (Firefox, Thunderbird, SeaMonkey, Sunbird)

http://hg.mozilla.org/mozilla-central/rev/99eb6c83cccf http://hg.mozilla.org/mozilla-central/rev/4a800d695ac9 http://hg.mozilla.org/mobile-browser/rev/16268e839812

bug 588874 is fixed. now if/when change this to true, UA is Mozilla/5.0 (Windows NT 6.1; rv:2.0b5pre) Gecko/20100827 Firefox/4.0b5pre Firefox/4.0b5pre dupe "Firefox/4.0b5pre" (with false : Mozilla/5.0 (Windows NT 6.1; rv:2.0b5pre) Gecko/20100827 Firefox/4.0b5pre) so this "general.useragent.compatMode.firefox" is no need ?

(In reply to comment #115) > so this "general.useragent.compatMode.firefox" is no need ? Correct, you won't need this pref if you're using Firefox.

(In reply to comment #116) > Correct, you won't need this pref if you're using Firefox. above UA is Minefield Hourly's UA. not Firefox.

(In reply to comment #117) > above UA is Minefield Hourly's UA. > not Firefox. Not for me. Mozilla/5.0 (X11; Linux x86_64; rv:2.0b5pre) Gecko/20100827 Firefox/4.0b5pre What is your general.useragent.compatMode.firefox pref?

(In reply to comment #118) > Not for me. > Mozilla/5.0 (X11; Linux x86_64; rv:2.0b5pre) Gecko/20100827 Firefox/4.0b5pre > > What is your general.useragent.compatMode.firefox pref? see above. (In reply to comment #115) > now if/when change this to true, UA is > Mozilla/5.0 (Windows NT 6.1; rv:2.0b5pre) Gecko/20100827 Firefox/4.0b5pre > Firefox/4.0b5pre > > (with false : Mozilla/5.0 (Windows NT 6.1; rv:2.0b5pre) Gecko/20100827 > Firefox/4.0b5pre)

Then what's the problem? Don't set the pref if you're on Firefox. :)

(In reply to comment #120) > Then what's the problem? Don't set the pref if you're on Firefox. :) don't use Firefox. use Minefield. now, Minefield normal/standard UA include "Firefox" Mozilla/5.0 (Windows NT 6.1; rv:2.0b5pre) Gecko/20100827 Firefox/4.0b5pre use Minefield with general.useragent.compatMode.firefox "true" Mozilla/5.0 (Windows NT 6.1; rv:2.0b5pre) Gecko/20100827 Firefox/4.0b5pre Firefox/4.0b5pre so no need to use "general.useragent.compatMode.firefox" dupe "Firefox/4.0b5pre" is the problem. (don't you think this is strange ?)

You don't need to use the pref with Minefield. It identifies itself as Firefox in the UA string now.

(In reply to comment #122) > You don't need to use the pref with Minefield. It identifies itself as Firefox > in the UA string now. so I said ""this "general.useragent.compatMode.firefox" is no need ?""

(In reply to comment #123) > (In reply to comment #122) > > You don't need to use the pref with Minefield. It identifies itself as Firefox > > in the UA string now. > > so I said ""this "general.useragent.compatMode.firefox" is no need ?"" Yes there is a need for it, read the first comments in this bug for why that pref exists. (Firefox is not the only (Gecko) browser on the planet)

(In reply to comment #124) OK, understand.

Filed bug 591536 to request hiding this compatibility pref when not needed. Firefox doesn't really need a pref to add a duplicate token to its UA. ;)

(In reply to comment #124) > Yes there is a need for it, read the first comments in this bug for why that > pref exists. > (Firefox is not the only (Gecko) browser on the planet) Though, actually, as this bug killed the "use Minefield/ in the UA to discover bad sniffing for 'Firefox'" intention that we had before (there was an actual bug for this that we *solved* by not having "Firefox" in the UA, I must assume Tech Evang for that kind of bad sniffing is dead and it's becoming ludicrous for any Gecko-based browser to not set compatMode.firefox - and actually, SeaMonkey is probably already pretty laughable for not having "Firefox" in its UA right now, if not for existing in the first place. That's how life goes, apparently.

(In reply to comment #127) > Though, actually, as this bug killed the "use Minefield/ in the UA to discover > bad sniffing for 'Firefox'" intention that we had before [...] I suppose you mean bug 588874?

(In reply to comment #128) > I suppose you mean bug 588874? Possibly, part of the same effort anyhow. In any case, I start to wonder why we don't just hardwire a |if (appname != Firefox) append "Firefox/x.y" to UA| thing and forget about .compatMode completely, as we obviously have given in to "Firefox" sniffers and acknowledge that "Gecko" should equal "Firefox" on the web.

(In reply to comment #129) > > I suppose you mean bug 588874? > > Possibly, part of the same effort anyhow. No, the motivation for this bug was bug 572650. I explicitly said that I don't want to replace Minefield here, so please cut the unrelated rants out.

It's not a rant, it's a request for going the full way and killing .compatMode given it's ludicrous for anyone not to set it now.

File a bug then.

(In reply to comment #132) > File a bug then. Good idea. Bug 591617 Submitted.

(In reply to comment #103) > product/productSub [Firefox/version] appName/appVersion [vendor/vendorSub] Sorry I missed that but, why should Firefox/version come before appName/appVersion?

(In reply to comment #134) > (In reply to comment #103) > > product/productSub [Firefox/version] appName/appVersion [vendor/vendorSub] > > Sorry I missed that but, why should Firefox/version come before > appName/appVersion? (In comment #98) > I think the order of tokens should be "[Firefox/version] productName/productSub > [vendor/vendorSub]". This would remain consistent with what Fennec currently > does, and makes sense to me from an order-of-importance standpoint. (If you > want to pretend to be Firefox-compatible, that should be the first thing we > advertise.) In other words, (if enabled) you want to announce Firefox first for compatibility for checking for bad sniffers that are looking for just what's in that one place or checking for Firefox first, then list the actual application next for those correctly looking for it. Not that this will always work (e.g. bug 594578). In an ideal world, this string is supposed to be listed in vague order of descending importance as well. So after "Mozilla/5.0" (aka "Mosaic killer" 5th-ish generation, not that it has any real meaning like this anymore) and the parenthetical with platform tokens and Gecko version, it should list Gecko, the base, before Firefox, the main project where browser development starts, before the final application built on top of this stack (regardless of how much of the underneath it uses). So for example, Firefox should be listed before SeaMonkey because whilst SeaMonkey is a separate project much of the code work that ends up in its browser comes from work done on Firefox, which is all based on Gecko.

> product/productSub [Firefox/version] appName/appVersion [vendor/vendorSub] and I think we were mixing up app and product at a few points in here.

Too bad I wasn't put on CC for this bug. Lightning very well needs support for setting an extra user agent part. While I understand the reasoning that not every extension should be adding something there, for Lightning it actually makes sense: The CalDAV Protocol, as well as the Provider for Google Calendar use http(s) as a transport. While in most cases we try to follow the spec closely, adding Lightning/1.x to the user agent will allow especially CalDAV servers maintainers that extensively use Lightning as a client to add special casing for Lightning. Is there any way left an extension can add user agent parts?

(In reply to comment #137) > Is there any way left an extension can add user agent parts? No, but you can set a custom HTTP header: https://developer.mozilla.org/en/Setting_HTTP_request_headers

We customize firefox for use by a rich client application used by our franchises. If the useragent doesn't contain our custom string then it can't get through our firewall and forces the user to download the custom client. 4.0 completely breaks that paradigm, although we will probably be using Google Chrome for our next client instead of 4.0 because the performance is so much better.

(In reply to comment #139) > Google Chrome for our next client instead of 4.0 because the performance Firefox 4 will be on par with it or likely faster, when complete. See arewefastyet.com for up-to-date graphs comparing development builds of Mozilla, Apple, and Google JavaScript engines. Mozilla just recently took the top place on SunSpider and further optimizations are in the works, so Firefox 4 will be fast in comparison to Firefox 3.x, by a noticeable amount. (and not just in terms of JS) Note the whiteboard at the top of this bug, by the way: [parity-IE][parity-safari][parity-chrome][parity-opera] Google Chrome doesn't let you customize the user agent string either, in fact current or future versions of all 5 major browsers don't allow arbitrary user agent string alterations like we used to. > We customize firefox for use by a rich client application used by our > franchises. If the useragent doesn't contain our custom string then it can't > get through our firewall and forces the user to download the custom client. > 4.0 completely breaks that paradigm Unfortunately, it's a tale as old as time: something validly useful to a few was abused to hell by others and had to be removed. Sorry, but it happens. You do, however, have alternatives. One route would be to use a custom HTTP header line (comment 138) instead of amending the UA string. In general, this is the advised route, and it's also best if you can set it to only send the header when needed and not to every site you vist. (or send it always, but have the firewall strip it, if possible) https://developer.mozilla.org/en/Setting_HTTP_request_headers

You can change the useragent in chrome with a command line switch and I guess you can do the same in firefox with the general.useragent.override setting. FYI I just tested FF4 beta 6 and Chrome is almost 6 times faster for partial page rendering. A partial update that takes 1.7 seconds to render in Firefox 4 takes 300ms in Chrome. I'm seeing about 20% better performance for full page renders as well. I could send you a Fiddler capture if interested.

(In reply to comment #141) This is off-topic for this bug. In any case, I should note that 4.0 beta 6 is really old now and the new JavaScript engine overhaul will first be in the long delayed beta 7 and later. The beta 6->7 jump is a big one for JavaScript as well as other performance areas, and again, things aren't done yet. Again: off topic.

Bugs are not a place to evangelize, for or against Mozilla products. Don, if you can provide a testcase of a page that has that much of a speed difference I know a bunch of people that would be interested in digging into it. Can you file a separate bug on that?

I have to say that I was disappointed to discover this preference had been removed in Firefox 4; it was something we had been using in my company to switch builds of our webapp - you could get to a particular (older or unreleased) version of the webapp (useful for finding bugs, confirming regressions, and so on) by switching your "general.useragent.extra.ourAppName" key to point at a particular build number. Just to make sure I understand correctly, this is now something we can only change by overwriting the entire UA string via "general.useragent.vendor"?

(In reply to comment #144) > it was something we had been using in my company It's always a bit annoying when a feature that may have some legitimate uses is abused and misused so much it needs to be removed, but the bad vastly outweighed the good here. > Just to make sure I understand correctly, this is now something we can only > change by overwriting the entire UA string via "general.useragent.vendor"? To completely change the UA string via a preference, set the new UA string in "general.useragent.override". The vendor UA prefs were removed in bug 591573.

> in my company to switch builds of our webapp - you could get to a particular > (older or unreleased) version of the webapp (useful for finding bugs, > confirming regressions, and so on) by switching your > "general.useragent.extra.ourAppName" key to point at a > particular build number. Tip: Use a different URL for the other version, e.g. http://yourcompany/app/3.4/ or http://yourcompany/app/?version=3.4

>"It's always a bit annoying when a feature that may have some legitimate uses is abused and misused so much it needs to be removed, but the bad vastly outweighed the good here" That sound like an opinion. Can you give some examples as to how this was abused and what problems it caused for web servers? I'm also very disappointed that this was removed.

(In reply to comment #147) > That sound like an opinion. Can you give some examples Comment 0 and comment 16 contain reasoning and examples.

How would you handle this use case without being able to modify the useragent. We have a web app that in order to use requires a firefox extension that we built. If the extension is not allowed to append to the user agent string, How does the web site detect that this firefox extension is installed and route the user to download page if not.

You can define a property on the global window such as "window.DonsExtensionInstalled". See http://daviddahl.blogspot.com/2011/05/extend-web-custom-window-properties-are.html for some instructions.

Better yet, set an HTML attribute on the documentElement. You can do that in a tablistener.

Or read some of the comments above that already suggest setting a custom HTTP header: https://developer.mozilla.org/en/Setting_HTTP_request_headers

Is the useragent string considered to be a http header parameter. Couldn't you use the technique of adding a custom http headers to read the current useragent string and append a value?

Please don't do that. The whole point of this change was to discourage such use. You could use one of the suggestions above, or add your own HTTP header. In any case, you should only do so for requests sent to your domain. This is easy to implement as extension code, and basically lets you implement it however you want since you're not affecting requests to other sites.

If anyone comes up with an example of how to modify the UA string using the custom HTTP header method, I'd appreciate a point in an aside - I know Dan is saying "Don't do that", but I just find comment #0 and comment #16 to be...not valid concerns for me. In our environment, we ONLY modify the UA string in a certain Firefox profile that is used specifically for hitting this internal server, and I am loathe to spend hundreds of man-hours re-engineering a new way to do this particular task as suggested by comment #146. I hate the idea of going all the to the "general.useragent.override" level, as that means throwing away the current wonderful UA data; but I spent about 3 hours trying to consume https://developer.mozilla.org/en/Setting_HTTP_request_headers and I still don't understand what the hell nsIHttpChannel is or how I'm supposed to use any of the example code given at that site. So if you have something I can take as a working example and modify the particular HTTP header I need to get sent by appending my strings to the UA, I would be very grateful.

(In reply to comment #155) The code in this Add-on SDK-built extension adds an "X-Bugzilla" header to all requests going to bugzilla.mozilla.org: https://builder.addons.mozilla.org/addon/1004701/latest/ If your extension isn't built with the Add-on SDK, you won't be able to lift the code directly, but the same principles should apply. (You'll just have to handle the register and unregister stuff yourself. And you'd use "Components.classes" and "Components.interfaces" instead of "Cc" and "Ci".) Feel free to take that code and tweak it to test for your site and add your desired header.

This comes just as I am trying to test some redirect code on my server which will be based on certain spiders' user agents. I had set a made-up testing string to add to my UA so I could see what the spiders would see before actually letting it loose on them, and was close to tearing out hair figuring out why the code wasn't working before finally figuring out the expected addition to the UA string wasn't there. So what are my alternatives in this use case, besides using an older version of the browser? Is anyone maybe working on an add-on to restore this functionality? It would be most welcome. Perhaps if it were limited to appending strings to the UA string on a site-by-site basis that would be acceptable to the high muuckety-mucks?

In reply to comment #157: I think the high muckety-mucks, as you call them, won't budge. IIUC the only way to change the user-agent string now is the preference general.useragent.override, which (IIUC) is intentionally not remembered from one session to the next. The "User Agent Switcher" extension may help you, but in Firefox 7, SeaMonkey 2.1, or later, you may have to override compatibility (e.g. by means of the "Add-on Compatibility Reporter" extension) to use it.

I understand the need to prevent abuse by rogue plugins leading to the desire to stop it. That's all well and good. So instead of allowing that to happen by default, as was the case, why not just disable use of the .extra.* by default, but allow those of us that want to keep it to turn it back on. Our entire company also relies on being able to "add something" to the UA that our webapp's client-side code can see via navigator.userAgent and act upon. No plugins involved. Simply a way of telling the running code to behave this way or that way. As a result of this "bugfix", we can't upgrade beyond 3.6 for Firefox users. I'd like to be able to say the following to them : "well, it was a much discussed change by the Firefox developers - a lot of arguing on either side, and as a result of that amount of disagreement, they naturally decided to disable that feature by default, but for those of us who aren't troubled by misbehaving plugins but still want to add stuff to the UA, we simply put a tick in a checkbox somewhere". Or we edit a resource file (.ini or .js), or go into about:config somewhere. But no, I have to tell them to uninstall Firefox and reinstall 3.6 and turn off updates. Super. ...and no, general.useragent.override isn't a good alternative...

I fully agree with your analysis Here’s a little trick I learned that might be of help In the firefox installation directory there is a file called application.ini. It contains the application name and version number and these two values are added to the useragent (they weren’t in previous versions). We changed the Name value so that firefox effectively becomes a custom app and doesn't conflict with other installations of firefox. If you add text after the version it will get appended to the useragent so you can edit it as shown below . This is actually better than overriding the useragent since the useragent will continue to correctly identify the operating system. Note that this will disable automatic updates. [App] Vendor=Mozilla Name=Firefox Version=4.0 myValue BuildID=20110303194838 SourceRepository=http://hg.mozilla.org/releases/mozilla-2.0 SourceStamp=5f8f494a4c29 ID={ec8030f7-c20a-464f-9b0e-13a3a9e97384}

If you edit it as follows the useragent will Contain MyClient/7.0 Firefox/6.0 [App] Vendor=Mozilla Name=MyClient Version=7.0 Firefox/6.0 BuildID=20110303194838 SourceRepository=http://hg.mozilla.org/releases/mozilla-2.0 SourceStamp=5f8f494a4c29 ID={ec8030f7-c20a-464f-9b0e-13a3a9e97384}

(In reply to Don Kleppinger from comment #161) Neat trick. An alternate to sticking the Firefox/version in after your version would be to just change your name and version to whatever you're calling it and then just flip the "general.useragent.compatMode.firefox" pref in about:config to add the Firefox/version part automatically. (i.e. rebranding your version and stating Firefox compatibility with the built-in pref instead of mixing it in with the rebranding)

Thanks for that info. I wasn't aware of that setting.

Thanks Don for your replies in comments 160 and 161. I haven't tried it yet, but it looks like it would achieve exactly what I need. It's unfortunate that it would prevent auto-updates. @Dave Garrett, I didn't quite follow what you were saying about the compatMode. Could you (or someone) elaborate a little on that? Ultimately I (and others it seems) want to be able to add/remove something to the user agent on demand. What we change it to may just need to be there for a few hours, or could be a few months. Am I right in assuming that any change to Name or Version will result in it being effectively orphaned and not treated as a "regular firefox" and therefore auto-updates will become disabled because the updates would be marked as being targeted for Name=Firefox, Version=X.X and anything that deviates from that gets non-updated. I wish there was a shortcut parameter (like --ProfileManager) that could allow you to send in something that would be added to the useragent. That would be the dream situation for me. I'd just tell users to duplicate their Firefox shortcut as many times as needed, and edit the shortcut for each one and add --UserAgentAddition=SomeString and they then just choose the shortcut they need depending on their needs. We achieve this on IE with .reg files that target the PostPlatform in the registry, quick and easy.

(In reply to Dee44 from comment #164) > @Dave Garrett, I didn't quite follow what you were saying about the > compatMode. Could you (or someone) elaborate a little on that? "general.useragent.compatMode.firefox" is a preference that can be toggled in about:config that simply adds Firefox/version (e.g. Firefox/6.0 for a Gecko 6.0 based application) to the user agent string. The pref was added in this bug alongside the removal of support for old methods that were sometimes used to do this. SeaMonkey and Mobile Firefox, for example, have this on so that websites with poor UA sniffing will see Firefox if they're looking for just that instead of Gecko or the application's name and thus send the correct version of the page for the browser. If enabled in Firefox it will add a duplicate Firefox/version entry to the UA string. If enabled in an install rebranded to no longer say it is Firefox using the method above, it will add the Firefox/version in addition to whatever else was specified. The problem with adding arbitrary things to the UA string is that even if it's done without breaking things (which is sadly not guaranteed), the browser is still sending the modified UA string to every site it visits (again, possibly breaking things, even if it shouldn't). Generally if you need to identify clients specially you should use an extension to identify itself to only the things that need it. As stated above, you can do this with custom HTTP headers [1] to be read by the needed server(s) or adding something to the DOM [2] that can be read by the needed page(s). [1] https://developer.mozilla.org/en/Setting_HTTP_request_headers [2] http://daviddahl.blogspot.com/2011/05/extend-web-custom-window-properties-are.html (comment 150) Unfortunately, as has been stated above, the abuse of the removed capability dwarfed valid use-cases and caused so many different problems that it needed to be removed. Also note that no other browsers have quick prefs like these. (as you say, IE can be modified via the registry in a manner that sounds similar to what Mozilla has in an INI file) If people are still having troubles with this, I think it might be a good idea if someone were to write a simple generic extension to implement these two methods that could be reused by others. Better examples on MDN would at least be a start if anyone who has done something like this would like to do so.

I have produced a simple generic example for someone that wants to modify the http headers. Feel free to contact me to receive it.

(In reply to Michael Kaply (mkaply) from comment #166) > I have produced a simple generic example for someone that wants to modify > the http headers. Feel free to contact me to receive it. The current docs are apparently somewhat confusing so even as a starting point that would probably be helpful. Have you considered posting it to AMO?

> The current docs are apparently somewhat confusing so even as a starting point that would probably be helpful. Have you considered posting it to AMO? It's not really appropriate for AMO since it's really just a skeleton app that shows how to modify the header. (It works as is, but that's probably not the right medium.) I'll try to do a blog post on it in the next week or so and put the add-on there. Of course if any company needs this done, I am available for hire :)

(In reply to Dave Garrett from comment #162) > (In reply to Don Kleppinger from comment #161) > Neat trick. It's pretty much the opposite of a neat trick. application.ini isn't used solely for the UA string and messing with it can (probably will) break other stuff.

(In reply to Dão Gottwald [:dao] from comment #169) > (In reply to Dave Garrett from comment #162) > > (In reply to Don Kleppinger from comment #161) > > Neat trick. > > It's pretty much the opposite of a neat trick. application.ini isn't used > solely for the UA string and messing with it can (probably will) break other > stuff. You left us no choice by removing a much-used feature just because in some cases it was being abused. For those of us unaffected by that abuse, we're left without a feature we relied on. The simple answer there, which most developers I know would think of, is to flip the default for the feature so that it's disabled. The malicious (or stupid) extensions would get no access to it, but the user that wants to keep that highly useful feature can simply turn it on. A warning could be display while turning it on if necessary. The 99% of users who don't use it won't notice you've even disabled it by default but things just got better.

(In reply to Dee44 from comment #170) > (In reply to Dão Gottwald [:dao] from comment #169) > > (In reply to Dave Garrett from comment #162) > > > (In reply to Don Kleppinger from comment #161) > > > Neat trick. > > > > It's pretty much the opposite of a neat trick. application.ini isn't used > > solely for the UA string and messing with it can (probably will) break other > > stuff. > > You left us no choice by removing a much-used feature just because in some > cases it was being abused. For those of us unaffected by that abuse, we're > left without a feature we relied on. > > The simple answer there, which most developers I know would think of, is to > flip the default for the feature so that it's disabled. The malicious (or > stupid) extensions would get no access to it, but the user that wants to > keep that highly useful feature can simply turn it on. A warning could be > display while turning it on if necessary. The 99% of users who don't use it > won't notice you've even disabled it by default but things just got better. Exactly In our case we have re-branded firefox for use with our internal web application. The users don't even necessarily know they are running firefox or even know that it's a browser because all menu's, buttons, bars and chrome have been removed by a custom extension and they can't navigate anywhere outside of the web app. We needed the useragent setting so that the browser can be redirected to our custom client download page if they try accessing the web app url with the regular flavor of firefox. Changing the application.ini file make it so our client doesn't interfere with any other version of firefox the user might have installed.

I don't know what all the fuss is about. If you tell your extension to modify the request header containing the user agent in the "http-on-modify-request" observer event, then you should be able to access it from your internal site (note this doesn't change how javascript thinks of the user agent, it will only be in the request header). I can well imagine this would be rejected by AMO unless there is due reason to do so, but if the extension you are using is internal only anyway, why not?

Really sorry such a useful feature was removed, and no any real alternatives. Override the whole UA (which smashes everything) is totally different from appending a string to the UA. Adding a HTTP header? that's apparently not ideal. If you just worry about it might be abused, why not just add some restrictions like total numbers/length it can be appended?

This blog is an example why the feature is very useful: https://blog.kloud.com.au/2014/11/06/implementing-adfs-v3-0-forms-authentication-in-mixed-environments/

(In reply to Philipp Kewisch [:Fallen] from comment #137) > Too bad I wasn't put on CC for this bug. Lightning very well needs support > for setting an extra user agent part. While I understand the reasoning that > not every extension should be adding something there, for Lightning it > actually makes sense: > > The CalDAV Protocol, as well as the Provider for Google Calendar use http(s) > as a transport. While in most cases we try to follow the spec closely, > adding Lightning/1.x to the user agent will allow especially CalDAV servers > maintainers that extensively use Lightning as a client to add special casing > for Lightning. > > Is there any way left an extension can add user agent parts? Actually, even today in 2017, IE v11 and Microsoft EDGE support adding a custom string to useragent by modifying registry. There are many use cases the .extra.* feature is very useful, for example, in DHCP environment, a unique string could be added to UA for each machine to track it(even when the IP addresses is changing). To prevent the abuse of the .extra.* feature, probably the right way to do it is to add restrictions like the total number of .extra., the max length of .extra. Removing a useful feature just because of afraid it might be abused, is something like stop eating just because of afraid of choke. Is it possible to create a new bug and restore the feature?

> a unique string could be added to UA for each machine to track it(even when the IP addresses is changing). This is not a proper use. The UA string identifies the browser implementation, not the user or machine. > Is it possible to create a new bug and restore the feature? This was removed 6 years ago. It is very unlikely to be restored.

(In reply to Ben Bucksch (:BenB) from comment #176) > > a unique string could be added to UA for each machine to track it(even when the IP addresses is changing). > > This is not a proper use. The UA string identifies the browser > implementation, not the user or machine. > > > Is it possible to create a new bug and restore the feature? > > This was removed 6 years ago. It is very unlikely to be restored. Thanks Ben. It seems to me that one of the reason the feature was removed was related to the Microsft Blog, but there might be misunderstanding of the statement of Microsoft (since IE 11 and EDGE still supports adding custom strings in UA). Some other use cases, if a web server doesn't see a secret custom string in UA, it knows it is very likely a request from a browser on a machine not owned by the organization (though it shouldn't be used as a way for authentication); some malware spoof UA like IE or Firefox, if a custom string can be added to UA, sometimes it can be much easier to differentiate a request from a real browser and a malware binary. I think one of the reason Firefox is great is that it gives users flexibility. So "flexibility" (giving users the options) probably is always better than "no, it's not supported". If a feature might be abused, probably it should be reviewed and prevent/mitigate the abuse instead of just removing it. Microsoft CD/USB Autorun has been abused for many years, but it is still supported :) As you can see in this thread, there are many people against removing the feature, that probably tells there are requirements for similar features.