Mozilla should offer a user pref to send the HTTP referrer header (a) always, (b) never, or (c) only if the requested resource is at the same server as the originating resource. Such an option is provided by iCab, and is useful for those who are more concerned with their own privacy than with the ability of Web site administrators to tell where their visitors are coming from.

The pref is already there: pref("network.sendRefererHeader", 2); // 0=don't send any, 1=send only on clicks, 2=send on image requests as well Are you talking about a UI? If so, please adjust the summary.

If `// 0=don't send any, 1=send only on clicks, 2=send on image requests as well' accurately describes the pref, then it's not what I'm asking for. This RFE is not asking for the Referrer header to be sent based on the method of requesting the resource, but based on the relation of the resource to the place it was requested from.

http bugs to "Networking::HTTP"

not critical for 0.8

*** Bug 69046 has been marked as a duplicate of this bug. ***

An UI for this preference would be very nice, I didn't know about it until my bug was marked a duplicate (ok, I couldn't spell referrer when I searched if my bug was already submitted...).

> I couldn't spell referrer You're not the only one - see the HTTP spec :).

This enhancement cannot be solved from within necko. It needs an owner outside of necko... perhaps in the uriloader? -> mscott? taking this off the radar for 0.9 since there is no owner yet.

I have an UI for this up and running. Take a look at my screenshot. I will dump that after this comment. I can check-in, a.s.a.y.w.i

RFC 2616 strongly recommends us to fix this bug (section 15.1.3): Because the source of a link might be private information or might reveal an otherwise private information source, it is strongly recommended that the user be able to select whether or not the Referer field is sent. For example, a browser client could have a toggle switch for browsing openly/anonymously, which would respectively enable/disable the sending of Referer and From information.

can CAPS help?

timeless, the Center for Analysis and Prediction of Storms <http://www.caps.ou.edu>? Please provide a URI.

http://bugzilla.mozilla.org/describecomponents.cgi?product=Browser Security: CAPS

> Security: CAPS OPS! /me hides.

I think that this RFE is more about extending the current pref settings because (a) always = pref("network.sendRefererHeader", 2); (b) never = pref("network.sendRefererHeader", 0); in the current implementation. So only (c) is missing at this moment, right? What about adding pref("network.sendRefererHeader", 3); for this??

mozilla sends file:/ in Referer One solution would be sending for the URL http://www.server.com/path/file.ext Referer: http://www.server.com/path/ This doesn't causes problems with broken servers and maintains user anonymity.

eperez@dei.inf.uc3m.es: the problem that we currently send referers from file:// URLs seems to be covered by bug 83038.

*** Bug 86949 has been marked as a duplicate of this bug. ***

> 1=send only on clicks > 2=send on image requests as well I don't see much point in differenciating between 1 and 2 there. Idealy, the options would be: Always send Don't send cross-site (needs better wording for UI) Never send This might want to go next to the User-Agent disable, UI wise, since they're both privacy affecting headers. (bug 92716)

A couple more options for this would be nice: 1) Have it "lie" and always show the top level site as the referrer (i.e., if you request http://www.mozilla.org/foo/bar.html it'll send http://www.mozilla.org as the referrer) This would help fix some (but certainly not all) sites that refuse to work properly if they don't like the referrer you send. 2) Be able to set these options (always/never/lie) on a per site basis, similar to the other security stuff. Ideally it'd let you set things based on either the site you are going to, or coming from. I'd certainly love to disable referrer when I'm coming _from_ google, I don't want sites knowing what my search terms were that caused me to visit their site. That kind of information is just (ab)used for them to figure out how to make their site rank higher on google. #2 would be much more work to implement, but perhaps #1 would be pretty easy and could be slipped in sooner (even without a UI to control it would be a wonderful option to have, and much more practical than trying to browse without referrer totally disabled)

Here is some back-end code (no GUI) that I wrote to augment the current network.http.sendRefererHeader preference settings, and also addresses some of the above suggestions (along with a few of my own ideas). Note that the preference item isn't 'network.sendRefererHeader', as mentioned above...I think that is a Netscape 4.x-ism. My patch adds three new levels to sendRefererHeader, and the abilities to send referrers only to the same domain and/or fake the referrer: * "Send no referrer to 3rd party servers" (network.http.sendRefererHeader = 4). * "Send a fake referrer to 3rd party servers" (network.http.sendRefererHeader = 5). * "Always send a fake referrer" (network.http.sendRefererHeader = 6). (the descriptions need rewording, but hopefully you get the idea) An example of sending a fake referrer is say you are on http://www.example.com/home.html, which loads an image from http://www.foo.com/img.jpg. Normally, foo.com would be sent the referrer 'http://www.example.com/home.html'. Using sendRefererHeader levels 5 or 6 (send fake referrer to 3rd parties), foo.com would instead receive the referrer 'http://www.foo.com', or nothing at all with level 4 (send nothing to 3rd parties). www.example.com and images.example.com would get the correct referrer with levels 4 or 5, or just 'http://www.example.com' at level 6. Hopefully these three settings are flexible enough to maintain privacy, without blocking access to servers that enforce referrer rules to block off-site access. While I typically block referrer altogether, I find level 5 (send fake referrer to 3rd parties) particularly useful, since it sends the valid referrer to servers in the domain you are visiting, and a valid looking referrer - without providing any useful information - to 3rd party servers.

Here is some back-end code (no GUI) that I wrote to augment the current network.http.sendRefererHeader preference settings, and also addresses some of the above suggestions (along with a few of my own ideas). Note that the preference item isn't 'network.sendRefererHeader', as mentioned above...I think that is a Netscape 4.x-ism. My patch adds three new levels to sendRefererHeader, and the abilities to send referrers only to the same domain and/or fake the referrer: * "Send no referrer to 3rd party servers" (network.http.sendRefererHeader = 4). * "Send a fake referrer to 3rd party servers" (network.http.sendRefererHeader = 5). * "Always send a fake referrer" (network.http.sendRefererHeader = 6). (the descriptions need rewording, but hopefully you get the idea) An example of sending a fake referrer is say you are on http://www.example.com/home.html, which loads an image from http://www.foo.com/img.jpg. Normally, foo.com would be sent the referrer 'http://www.example.com/home.html'. Using sendRefererHeader levels 5 or 6 (send fake referrer to 3rd parties), foo.com would instead receive the referrer 'http://www.foo.com', or nothing at all with level 4 (send nothing to 3rd parties). www.example.com and images.example.com would get the correct referrer with levels 4 or 5, or just 'http://www.example.com' at level 6. Hopefully these three settings are flexible enough to maintain privacy, without blocking access to servers that enforce referrer rules to block off-site access. While I typically block referrer altogether, I find level 5 (send fake referrer to 3rd parties) particularly useful, since it sends the valid referrer to servers in the domain you are visiting, and a valid looking referrer - without providing any useful information - to 3rd party servers.

> An example of sending a fake referrer is say you are on > http://www.example.com/home.html, which loads an image from > http://www.foo.com/img.jpg. Normally, foo.com would be sent the referrer > 'http://www.example.com/home.html'. What, if I am on <http://www.example.com/home.html> and click on a link to <http://www.foo.com/home.html>? Normally, this would count as 3rd party. Also in your new code? Generally, thanks for the code! (Didn't have a chance to look at it yet.)

Yes, that would also be considered a 3rd party request. So when you clicked on the link at example.com, foo.com would receive the referrer 'http://www.foo.com' (assuming you were using one of the modes that sends a fake referrer to 3rd parties), or no referrer at all. A more concrete example of why you might want to block or alter 3rd party referrers (as given above by slice1900) is doing a Goggle search. Since Google uses GET to handle the search form, the search keywords are part of the URL and thus part of the referrer. Maybe you don't want that information being sent out every time you click on a search result. Another possible scenario is let's say you log into a website which puts some sort of personally identifiable information in the URL (username, account number, etc.). At this same site, doubleclick.net as a banner ad (or a web bug...) displayed. Unless you are blocking 3rd party/all referrers, or sending a fake referrer to 3rd parties, doubleclick.net now has your personally identifiable information.

>Hopefully these three settings are flexible enough to maintain privacy, >without blocking access to servers that enforce referrer rules to block >off-site access. Unfortunately, the "fake referers" prevent servers from limiting off-site access. Take this example: earlier this month someone embedded one of my scenic photos in his signature on a web forum. It only displayed at 75x75, but he linked to an 800x550 image, so every time someone looked at a thread he posted in, they were downloading a 112 KB file from my server. So I adjusted my server to block access to the full images if the referrer didn't show my own site. My bandwidth use then returned to normal. (I could have just renamed the file, but then I'd have to do it again the next time.) The fake referer that sends the target site's own address would completely bypass this by making it appear that the embedded image in the forum was actually a request from my own server. If Mozilla were a more popular browser, and if this were a popular setting, I'd have the same problem all over again. Admittedly there are more foolproof (and complicated) ways to handle the problem, and the people most likely to use this setting would probably also be blocking third-party images, but I wanted to bring up an issue from a web developer's perspective. Just something to consider.

kelson, from a user's perspective: > So I adjusted my server to block access to the full images if the referrer > didn't show my own site. I do have referers off al the time, because they are an invasion to my privacy. So, since nothing != your site, you would block me, even though I visit your site normally. It is exactly this short-sighted behaviour of websites why the fake referer is useful. Be glad that some users to have referers on so that you at least know which page linked to your resources. Now, you can maybe do something against the linking resource. *That* is a legitimate use of referers. And the fake referer option doesn't diminish it. Compare bug 86194.

kelson@pobox.com makes a good case for the legitimate uses of referrer. Unfortunately, since Referer is user generated header, it is and has always been open to manipulation. So relying on it to protect your content is really just 'security through obscurity', since most users don't have the means to modify it. It is hard to disagree with your particular situation, but Mozilla has a strong history of favoring the user over the content provider, especially when it comes to the user's privacy. If it is any consolation, it is very unlikely that 100% of users would be blocking or faking their referrer headers, so you would still be able to track down the offending site and deal with it in some other manner.

Even with faked referrers the blocking described in comment #27 would still accomplish its purpose: the luser referencing your image might set the pref and view his own .sig, but the thousands of people reading his posts and involuntarily flooding your server are not likely to have done so.

I've updated the patch to now match on the same host, rather the same domain. I originally wanted to be able to match on the same domain, based on the mistaken belief that any host on the same domain would be controlled by the same organization. But this is clearly not a valid assumption, especially for countries that use a common second level domain, such as .co.uk. So now the check for a 3rd party server is based solely on comparing hostnames, similar to the handling of 3rd party cookies and images. Other than this change, and some minor code cleanup, the patch’s functionality is unchanged.

I've added a preference boolean, network.http.referrerSchemeOverride, to enable the user allow the sending of a fake referrer for URI schemes (file, resource, etc.) that normally prohibit sending any referrer. This allows items such as bookmarks to send a valid looking referrer, so you may be able to bookmark URLs that would otherwise be inaccessible due to a page's requiring a referrer header. Since the referrer sent is based solely on the target URL, there is no affect on user privacy.

Jeff, your patch has nothing to do with this bug, nor do I see a valid purpose in lieing to a server like that.

Jeremy - Since you haven't read the above comments, please do so now. Then let me know if you have any questions.

This patch should be sufficient to resolve this bug, since it includes both the referrer enhancements of my previous patches as well as the Preference user interface.

>Relying on [the referer header] to protect your content is really just >'security through obscurity', since most users don't have the means to modify it. Referer-checking is not about protecting your content from your users. It's about protecting your bandwidth from other web sites that are too lazy to copy your images onto their site. >If it is any consolation, it is very unlikely that 100% of users would be >blocking or faking their referrer headers, so you would still be able to track >down the offending site and deal with it in some other manner. If Mozilla gains a pref to lie about referers, the following will happen: 1. Shady porn sites will advise Mozilla users to configure Mozilla to lie about referers, and then use copies of an image hosted elsewhere. 2. The victim porn hosts will block Mozilla just like they block requests not containing referers. (Why do porn sites block requests without referers? IE contains a bug that allows sites to create links that do not send referers when followed.) Besides, lying about referers doesn't do anything more for users than "Send referers only within the same host".

> Shady porn sites will advise Mozilla users to configure Mozilla to lie about > referers Does anyone read text at porn sites? ;-P > IE contains a bug that allows sites to create links that do not send referers when > followed. Even in <img src> or just <a href>? > Besides, lying about referers doesn't do anything more for users than "Send > referers only within the same host". It allows to follow deep linking to HTML pages, even if the target site doesn't want to. As for images, what about this situation: A site hosts images on a different host than the HTML page (like images.aol.com). That image host blocks requests with no referer (bad site!) or with a referer that is not from known-good sites. Since the browser can't know the relationship between the 2 hosts, that is a potential privacy leak and it blocks the referer. No images. As a sidenote: While I usually don't like to care about dirty people like porn hosters, Beonex Communicator ships with a privacy-protecting default setting. But porn seems to be a much-wanted content type, so users will be upset, if half the porn sites don't work (I wonder why I didn't hear complaints yet). That's why a setting is needed that 1) protects the privacy to the maximum extend *and* *at the same time* 2) works with almost all sites. And I think the faked referer is a very good idea to achieve that, even though it might not be the nice way. Thanks again, Jeff, for contributing that code. If people disagree with the option given, you could just remove them from the UI, but leave the backend in place. I will gladly take the risk of Beonex Communicator being blocked. Anyways, can anyone review the patch? I'd do it, but I don't know that code part and are getting short on time, too. I'll do a quick review before inclusion into B.C., but that can't count as r=. mscott, any objections to reassigning the bug to Jeff?

> It allows to follow deep linking to HTML pages Whatever that is... Here's the deal. We should eaither send a correct referer, or none at all. If unscrupulous third-parties want to make a gecko browser that lies, fine. The only thing of concern in a legitimate browser is the privacy issue of whether or not to send a referer header across hosts/domains, which brings up the same problem from third party image cookie detection, how do you tell if a.b.c and d.b.c are third-parties. Depends how c and possibly b is administered. So if the pref is only to send referer to the same host (only garuntee), many sites may break image loading, if their images are on static.domain.com or some such.

Jesse Ruderman writes: > Besides, lying about referers doesn't do anything more for users than "Send > referers only within the same host". Sure it does. Say you are browsing www.example.com, which serves images from images.example.com and requires an valid example.com referrer. If you have the referrer set to 'same host only', images.example.com won't receive a referrer and would deny access. There are plenty of other "legitimate" reasons why one would want to send the modified referrer. There is simply no reliable way to send a referrer to only valid recipients, and as you can see, 'Same Host Only' is actually not much better than never sending a referrer at all. Whether you think it is ethically appropriate or not, sending a modified referrer is really the only way to maintain both the user's privacy as well as access to many websites.

Ben Bucksch writes: First of all, thanks for your support. > If people disagree with the option given, you could just > remove them from the UI, but leave the backend in place. > I will gladly take the risk of Beonex Communicator being > blocked. I have been thinking about this, and I'm sure that Netscape/AOL wouldn't include this patch in the next version of Navigator, for the valid reasons that some people have already given. I concur with your suggestion to remove the new referrer settings from the UI for some browsers, and leave it in for browsers (Beonex and hopefully Mozilla...) that favor the user's privacy over the convienience of content providers. It will be very easy to simply hide the new referrer setting radio buttons, and I'll probably address this in the next patch. > Anyways, can anyone review the patch? I'd do it, but I > don't know that code part and are getting short on time, > too. I'll do a quick review before inclusion into > B.C., but that can't count as r=. Actually, as of the major check-in for bug #124042 (UTF-8 URIs), this patch is likely completely broken. I have been waiting for the smoke to clear from the check-in before I update my code and go about reimplementing my changes. Nothing major, just switching string objects to those that support i18n characters.

Ok, I updated the patch so that it patches cleanly and works with the UTF-8 URI changes. I also added a variable to pref-referrer.xul to hide the options that send the modified referrer. So it is possible to ship a browser that only displays the standard referrer options (never, only on click, on clicks and images), as well as the "Same Host Only" option.

comments about the patch: Firstly, isn't the referer spelt referer? Secondly, <outliner> was renamed to <tree>. Instead of |nsCRT::strcasecmp(referrerHost.get(), host.get()) != 0| you should probably use |!referrerHost.Equals(host, nsCaseInsensitiveCStringComparator)| + nsIURI *refPtr = referrer; Personally, I don't usually like such pointers...

"referrer" is the correct spelling. The RFC incorrectly(?) used "referer".

Well, the question is, do we want to use the name from the RFC (the authoritative source about HTTP headers), or from the dictionary?

> Secondly, <outliner> was renamed to <tree>. Yeah, but only after I had already submitted my patch. Thanks for the reminder, though. > + nsIURI *refPtr = referrer; > Personally, I don't usually like such pointers... Any particular reason, in the context of how it is used in this patch? > Well, the question is, do we want to use the name from the RFC (the > authoritative source about HTTP headers), or from the dictionary? I see no need to perpetuate the spelling mistake. Referrer is the English descriptive name, and Referer happens to be the HTTP header name. If one were localizing the UI to a non-English language, they would translate "referrer" to the equivalent native word. Leaving it as "Referer" would confuse everyone, except for readers of the RFC, and I think they can figure it out...

>> + nsIURI *refPtr = referrer; >> Personally, I don't usually like such pointers... >Any particular reason, in the context of how it is used in this patch? well, no, not really, this line is probably ok

oh yeah, and why not move the frontend to xpfe/components/prefwindow/resources/.../? The backend code isn't part of the cookie extension, so the frontend shouldn't be either.

I am currently merging the patch into the branch, of course it bitrotted again.

- I moved the UI to xpfe/components/prefwindow, because (as biesi said) the backend lives in netwerk/protocol/http/ and the whole thing has nothing to do with cookies or wallet. - Added Groups to match the rest of the prefs UI - Devided the explanation text, partly so that we can hide the fake header text, if that UI is disabled. Also looks better IMO. - I think that "modified referrer" (previous wording) is by far to general. I used "fake referrer" now, which IMO describes it better. Also made clear that this violates the HTTP standard. - Added explanation what referrers are at all. Not sure, if it's clear enough or how it could be made clearer. - Cutted wording of some of the options, so they are all one-line now. - Changed the "schemeOverride" (other schemes) pref UI from checkbox to radios, so I can make clear, that no referrer at all is sent, if the option is disabled. (I was not sure, if it sends nothing or the normal referrer, if disabled; in fact, I had to read the source to find out.) BTW: I did not review the backend part, at least not completely. Requesting review.

-> default HTTP owner

This time for real.

Thanks for the UI help, Ben. One concern - I think the word "fake" is a little too strong of a word and has too many bad connotations. I'm not really happy with "modified" either...so maybe something like "privacy referrer", or "anonymous referrer"?

It's deliberately wrong information designed to look correct. From the POV of the site, we're making the referrer header completely useless to them, if that feature is enabled. I don't think that "fake" is too strong for a standard violation. "privacy" - too vage "anonymous" - It's so much not about anonymity. Only in very rare occasions do you reveal identity through the referrer (which would not already be known).

> It's so much not about anonymity. eh, It's not so much about anonymity.

I'm not sure I like this "fake referrer" feature. We should definitley allow fine-grained control over when referrers are sent, but I believe they should either be sent as correct values or not at all. I'm going to get some more opinions on that.

Mitch, I don't *like* it either, in fact I have a bad feeling, but I see no better solution, see comment 37. Does it help, if the UI for these options is disabled in Mozilla? Changing one constant gives this UI.

Privacy is very important, and a pref+gui to disable the referer is definately a good thing, which I would use myself... but lying about the referrer is going too far. Ommitting the referrer is privacy. falsifying the referrer is ... well, just lying. I think the sites that would block a blank referrer represent a *very small* subset of sites, and they ought to be dealt with as evangelism issues, not with a fake referrer header. Of course I guess I would also be happy if you make the text of the last two prefs in the gui BLINKING RED and add the words "Very Bad Karma" somewhere in the description :)

"3rd party" should read "third-party". What did P3P call it? The whole paragraph explaining what the pref does needs to be redone. I *know* what HTTP Referer: is, and I can't even make sense of that wording. Re: the bookmark section of the first screenshot. Do any browsers send a referer for a bookmark? This might be nice for sites to be able to know, if we could default such a thing to being on by default. Referer: X-Mozilla Personal Toolbar Bookmark Referer: X-Mozilla Bookmark if that doesn't violate the spec. I don't think that's much of a privacy issue, eaither. A normal cross-site referer is much worse, since it could show you as coming from pornos-r-us.com, etc. Also, I second/third/whatever the feeling that we not include a "lie" option.

> The whole paragraph explaining what the pref does needs to be redone. Feel free to do it. I couldn't think of a better wording. > Referer: X-Mozilla Bookmark > if that doesn't violate the spec It does: "The Referer[sic] request-header field allows the client to specify, for the server's benefit, the address (URI) of the resource from which the Request-URI was obtained" "The Referer field MUST NOT be sent if the Request-URI was obtained from a source that does not have its own URI, such as input from the user keyboard." (HTTP 1.1 spec, <http://www.w3.org/Protocols/rfc2068/rfc2068>, section 14.37 on page 130)

(this is a lot of deja vu from the prior comments, but...) 1. Providing the modified/fake/privacy/anonymous/whatever referrer is the only way to maintain both privacy and access to many websites. If you don't like it, there are the other options available. In general, Mozilla would only be "lying" to 3rd party sites, who shouldn't be receiving a referrer in the first place (see RFC 2616). 2. (Optionally) sending the referrer for bookmarks is again necessary for access to many websites. Incidently, since the bookmarks do have an URI, sending a referrer doesn't violate RFC 2068. Mozilla just never used to send a referrer because, since the URI's scheme was internal to Mozilla, it wasn't useful for the website. Also, sending the actual referrer or something like "X-Mozilla Bookmark" betrays the fact that the user has bookmarked the website - a fact that the user may not want to reveal.

I don't think it's useful to differentiate between links and images, because links can point to images, and because javascript+frames can imitate a link to an image. Also, images are "protected" using a referrer check far more often than links are. Proposed pref UI: When loading links and images, tell the server the referring [or linking] address: ( ) Never ( ) Only when the linking page is on the same server (o) Always That would be simple enough to understand quickly, and it would also be consistent with the radio sets for image and cookies prefs.

In response to BenB's comment 37: you don't need the ability to fake referrers. There's already a setting you can use as the default for Beonex that would respect Beonex users' privacy without breaking porn sites and without prompting sites to block requests with "Beonex" in the user-agent string. It's "Only [send the linking address] when the linking page is on the same server".

I think we should consider removing the "Never" option. Sending the referrer within a site does not hurt privacy because sites can keep track of what the user loads without using referrers. Not sending the referrer within a site breaks a lot of sites (not just porn sites) and many users would not figure out why the sites are broken.

In my opinion, the current patch is not of check-in-able quality. Briefly, the problems are as follows. (1) It uses an entire prefs panel, which is far too much interface for such an obscure feature. (2) It has three paragraphs of text at the start. This is a dialog, not a help file. Even one paragraph of text would be far too much. (3) It has no controls outside the group boxes, which is an accurate predictor of the group box widget being misused. In this case `Referrer' is separated from `From bookmarks and other non-web sources', as if `From bookmarks and other non-web sources' is not to do with referrers, when actually it is. I'm not picking on Ben in particular here; many contributors -- including Netscape employees -- make the same mistakes, but I point them out whenever I get the chance, and 35 wrongs don't make a right. (Filing bugs on other perpetrations of (1), (2), and (3) is left as an exercise for the reader.) In line with comment 0 and comment 64, I suggest the following: | | Scripts&Windows | | Tell Web pages _which page you came from | | |:Privacy:::::::::| | (send "Referer" information): | | | History & Cache | | (*) always ( ) if the same server ( ) never | I've been fine-tuning this design for the past two years (yes, the `Referer' spelling is deliberate, and no, I don't think I'm dumbing it down too much), so I'm unlikely to change it in response to Bugzilla comments. But if you think Darin and Tom aren't getting enough bugspam, you're welcome to try. :-)

'Web pages'... I don't usually think of a picture as a webpage, would you consider 'web server'.

> I don't think it's useful to differentiate between links and images, because > links can point to images I don't know the backend implementation, but the description of the pref is probably inaccurate in an attempt to be understandable (it is the same description in the default prefs file). I guess what really is being meant are "embedded objects". (I tried to make that clear in the tooltip.) Jeff, can you confirm that? It might be worth to be added, because I can control link clicks, but I cannot control loading of embedded objects (possibly from another server). > I think we should consider removing the "Never" option. No! It is "strongly recommended" by the HTTP spec. > "Only [send the linking address] when the linking page is on the same server". That doesn't work, if the linked images are on another server. I think, that's not unusual in that scene (Not sure, though; and I'm not in the mood to check it right now ;-P .) > (1) It uses an entire prefs panel, which is far too much interface for such an > obscure feature. Feel free to make a better design with the same options. > (2) It has three paragraphs of text at the start. This is a dialog, not a help > file. Even one paragraph of text would be far too much. First, the one to be used in Mozilla has one subheader and one paragraph of text. I agree that it's too much text and not understandable enough. I said: "Not sure, if it's clear enough or how it could be made clearer." and "Feel free to [re]do it. I couldn't think of a better wording." We can remove the subheader altogether. I like your wording. Suggestion: Mozilla (fakes disabled): ---- Tell Web servers which page you came from. This can harm your privacy by allowing third parties to see which sites and pages you visited just before them. +- Web ----- | o Never | o Always | o For all pages, but not for [embedded objects | images] | o Only if the same server +--- ---- Beonex Communicator (fakes enabled): ---- Tell Web servers which page you came from. This can harm your privacy by allowing third parties to see which sites and pages you visited just before them. Using one of the faked referrer options violates the HTTP standard, but can help you to protect your privacy, while still allowing access to sites which wrongly require the referrer header to be present. +- Web ----- | o Never | o Always | o For all pages, but not for [embedded objects | images] | o Only if the same server | o Send faked referrer, if not the same server | o Always send faked referrer +--- +- Bookmarks, emails etc.--- | o Never | o Send faked referrer +--- --- Legend: [either | or] > (3) It has no controls outside the group boxes, which is an accurate predictor > of the group box widget being misused. No, not in all cases. Option groups are grouped by group boxes (on some systems even technically), so if the option group is the only content of a dialog, just one group box is reasonable. Also, it looks much better (compare the screenshots). > In this case `Referrer' is separated > from `From bookmarks and other non-web sources', as if `From bookmarks and > other non-web sources' is not to do with referrers, when actually it is. Correct, and that annoyed me, too. I couldn't come up with a better name for the first group box back then. > I'm not picking on Ben in particular here; many contributors -- including > Netscape employees -- make the same mistakes, but I point them out whenever I > get the chance, and 35 wrongs don't make a right. (Filing bugs on other > perpetrations of (1), (2), and (3) is left as an exercise for the reader.) Yes, I am aware of that, that's a major part of the reason why I did it. I agree that our use of group boxes in the pref panel generally sucks, but consistency is also a value to consider and I think that the group box use is acceptable in this case.

> I don't think it's useful to differentiate between links and images, because > links can point to images The difference is that links are directly initiated by the user (e.g. clicking on an anchor), and images/embeds are usually automatically loaded when the page loads, so the user doesn't have the same amount of control over when the referrer is sent. I didn't create this distinction - this is a holdover from Netscape Communicator. With the new option of only sending the referrer to the same host, this distinction isn't as useful as it once might have been... I can't really think of a good reason to have both options from an end user's perspective. From a developer's perspective, some reasons might be that the code is already in place, it is nice to have a GUI that fully represents the available prefs options, and to maintain compatability with Netscape 4's referrer options.

Beonex Communicator 0.8-stable has been released with this patch, default 3rd party fakes. I saw no problem myself, and I got no complains.

This issue has significant ramifications regarding e-commerce. From the AuthorizeNet web site: "Effective May 28, 2002, Authorize.Net will require all Merchants who connect to the payment gateway via WebLink or ADC Relay Response to implement a Referrer URL on their account. ANY MERCHANT WHO FAILS TO MEET THIS REQUIREMENT WILL BE UNABLE TO PROCESS TRANSACTIONS AFTER MAY 28, 2002, UNTIL THEY ARE IN COMPLIANCE WITH THIS NEW SECURITY MEASURE. IF YOUR WEBSITE CONNECTS TO THE AUTHORIZE.NET PAYMENT GATEWAY VIA WEBLINK OR ADC RELAY RESPONSE AND YOU HAVE NOT SET A REFERRER URL ON YOUR ACCOUNT, ALL YOUR TRANSACTIONS WILL BE DECLINED AS OF MAY 28, 2002."

This issue has a significant impact on e-commerce. AuthorizeNet is one of the world's largest online credit card processors. From the AuthorizeNet web site: "Effective May 28, 2002, Authorize.Net will require all Merchants who connect to the payment gateway via WebLink or ADC Relay Response to implement a Referrer URL on their account. ANY MERCHANT WHO FAILS TO MEET THIS REQUIREMENT WILL BE UNABLE TO PROCESS TRANSACTIONS AFTER MAY 28, 2002, UNTIL THEY ARE IN COMPLIANCE WITH THIS NEW SECURITY MEASURE." As a web developer specializing in e-commerce sites, this issue is huge. Thanks.

Beau Bennett, bug 141164 is specifically about that site. There, you can also read that his site violates the HTTP standard by this measure.

Rather, bug 141641.

Someone implemented a toolbar to spoof the referer. You can find it at http://refspoof.mozdev.org/

Instead of using the option of "same server", how about "same domain"? That would keep user privacy intact while not breaking sites which require appropriate referrers to load images but host said images on other servers. It also resolves the "pages versus embedded objects" debate, because if the automatically-loaded item was hosted by someone else the referrer would be withheld.

> Instead of using the option of "same server", how about "same domain"? That > would keep user privacy intact while not breaking sites which require > appropriate referrers to load images but host said images on other servers. It > also resolves the "pages versus embedded objects" debate, because if the > automatically-loaded item was hosted by someone else the referrer would be > withheld. It is a great idea, but I don't think "same domain" can be made to work. See <http://bugzilla.mozilla.org/show_bug.cgi?id=55477#c31> for more details.

While comment 34 is true, unfortunately this means this referer setting may be quite unusable. Consider all the www.foo.com sites that have their images on images.foo.com. Many likely block no referers, so other sites don't use their images and steal bandwidth. Comment 34 also points out an even worse problem. Domain cookies can't tell where they are valid for. I have a cookie set for ".co.uk", which would let every .uk site share my data. Yikes. Do the RFCs say anything about this, at least for cookies? Would it be out of the question to include a listing of TLDs and how many levels of generality they have? com 1 us 1 uk 2 au 2 ca 1 # I think some are 3 levels too Probably impossible to manage, since any TLD could change without notice. Just an idea.

> While comment 34 is true, unfortunately this means this referer setting may be > quite unusable. Consider all the www.foo.com sites that have their images on > images.foo.com. Many likely block no referers, so other sites don't use their > images and steal bandwidth. Just to be clear, although at one point, there was a patch that had the "same domain" feature, the latest code does not have this feature for the reasons previously mentioned. This again points out why the fake referrer is so useful. If example.com loads images from images.example.com, and images.example.com needs a *.example.com referrer, the fake referrer will send "images.example.com" and the page will load normally. "Same Host" will result in blocked images in this case. That isn't to say that "Same Host" is a useless option - for most sites, it probably would work fine - at least as well as the same host only options for Images and Cookies.

> the fake referrer will send "images.example.com" and the page will load normally No, in fact that's less likely to load then no referrer. It would obviously have been faked, bacause "images.example.com" is unlikely to have links to those images. A direct request (no referrer) would much more likely be allowed. I imagine a lot of image sites are set to allow if no referrer, or if referrer = one specific site that should be linking in.

> It would obviously have been faked, bacause "images.example.com" is > unlikely to have links to those images. Indeed, that's what just came to me. That's a way to block people with fake referrers enabled :-(. Let's just hope that webmasters are not that hostile and engaged. > That isn't to say that "Same Host" is a useless option A slightly offtopic anecdote: I have once been hurt by the "same host" policy of the image blocker. I had the same host policy enabled and used ebay, where a lot of sellers link images on their own servers from the auktion pages on ebay. Of course, that image was blocked. I ended up buying a GameBoy game when I wanted to buy an N64 one. There was a tiny category name, which I overlooked. It would have been obvious, had I seen the image. Anybody needs WaveRace for GameBoy? ;-P

Something to consider here is that no scheme is going to be perfect. A given non-conforming or restrictive site will always choke on one anti-referrer policy or another. Therefore, schemes should be included that strike different balances of privacy enhancement versus obstacles to browsing. Let the user decide. (This is already what's happened, but "same host" is the only thing between all and nothing. You don't have to get rid of "same host", just add a couple more choices.) Could a "99%" version of a "same domain" scheme be offered? Drop the first segment of the two server addresses, and if what remains matches you go ahead and send the referrer. This is more inclusive than "same host", which means less obstacles to browsing. In most cases it will prevent referrers being sent when you switch sites, which enhances privacy. The exceptions for this scheme are: - If you have loaded a page at a URL that contains only the domain instead of a real server name (http://ebay.com), and the link you click also only contains a domain (http://macintouch.com), the referrer will send. - Sites that use sub-domains extensively and require referrers to load content may choke. If a page at abc.fu.bar.com embeds an object from xyz.zoo.bar.com, no referrer is sent. - When a site is expecting a specific referrer from a site in a different domain (see comment 72), a failure would occur with any referrer-blocking scheme. Anyway, I believe the exceptions I listed will amount to a very small proportion of browsing situations. Because the policy is less restrictive than "same host", it means less user compensation is required. Any coders care to actually build it? (I'm a non-coding freeloader bum. :-)

There's a lot of possible scenarios being listed here. I think we need to get back to basics here and consider what it is we really need. Thinking in terms of privacy only, here are some situations I definitely do not want a valid referrer sent: * Coming from a file: or any other non http/https address * Any time I type a URL in the address bar * Any time I select a bookmarked site * Any time I click a link from google or other search engines * Any time I select something from history (including the back button) Handling the first three is trivial, just send no referrer at all. The fourth is more difficult, and may require a blacklist of domains for which to never send a valid third party referrer (just send nothing instead) To handle the fifth properly, I'd want the referrer that was sent when that page was originally accessed resent (is it currently kept in history? could it be?) None of these should result in any problems at all. Where we run into problems is providing a global ban against third party referrers. Perhaps we ought to see about getting the items above (especially the first four) taken care of, then move on from there. If we wait to get a perfect and mutually agreeable scenario (assuming one was possible) before anything is done, we'll be leaking a lot of data about our browsing habits that web sites have no business having. I'm most particularly concerned about websites using google search terms (which are encoded in the referrer link) to gather information I'd rather they not have. Personally I'm not so worried if a site widgetinfo.com has a link to widgetsales.com, and they know that's how I found out about widgetsales.com. Privacy is good when the information can be used against you or in a way you don't like, but privacy for no reason is pointless (anyway, there can always be a paranoid 'no referrer ever' option for the tinfoil hat crowd)

Regarding #4: If someone wants to block search engine results from showing up, but doesn't mind letting the site know where they came from, wouldn't it be easiest to just strip off the query string? That way the site knows someone found it on Google, but doesn't know what the person was searching for. No need to make a blacklist, try to guess whether a site is a search engine, or anything else. And if they don't want the site to know they came from Google, well, thats what the "block all third-party referrers" option is for.

I think we should let the user choose preferences like this: ------------------------------------------------------------ |Referer Prefs | | Case1 Case2 Case3 |Action1 (o) (o) () |Action2 () () () |Action3 () () (o) where the user can choose an action(i.e. fake referer) for each case (i.e. same server page), and allow the user to assign an action to multiple cases.

Re: comment 79. That cookie problem (domain = .co.nz) has been know for a long time and it is unsolvable -- at least not without breaking major sites. See bug 8743 for a lengthy treatise on this.

Thanks for the pointer, Morse. The pref discussed there-- network.cookie.strictDomains--seems to have been removed though. It seems rather hopeless eaither way, though. Same would apply to this bug.

That pref has not been removed. See nsCookies.cpp line 66 and line 1244.

This is a comment from a top site regarding this feature: Our website security systems verify that access is allowed by checking the referrer. If this is left in the user’s hands, the user will probably not choose to send it, and will then be unable to be authorized for access by a number of systems. This feature would only be useful if it was either always on, or always on by default and required a little extra sophistication on the user’s end to find and change that setting.

> Our website security systems verify that access is allowed by checking the > referrer. According to the HTTP spec, such behavior is broken.

According to common sense, that's broken. Any malicious user can send a false referrer. (wget has a command-line option for it.)

mike, jonas: yup of course it's broken for websites to rely on the HTTP referrer for anything meaningful, but _many_ sites do depend on it. more importantly, they depend on browsers having a consistent referrer policy. that means, if Mozilla doesn't send a referrer when IE sends a referrer, then Mozilla is going to appear broken in the eyes of many users (they won't blame the websites), and we really can't afford that IMO.

There was never any question that a pref that never sends a referer would break many many (broken) sites. Even the pref setting to send referer to same site will break some sites, as I described in comment 38. Netscape will very likely not want to expose UI for this. Mozilla may not as well. But I, an apperantly some others here would at least like a 'same host only' referer pref, and will be able to deal with the sites that break. ANYTIME we do ANYTHING different from the standard (ie, what IE does), there's something somewhere that will break. Just look at all the evangelism bugs. Limiting sending referer is a balance between breaking things and privacy. Unfortunately it seems too many people here want that balance at too many different places. Maybe Mozilla isn't the place for all these options. Personally I'd just like 1 setting of not sending it on cross-host PAGE loads (but still on images, since there's not much privacy lost there, but a high page break probability). But I'm willing to concede maybe this is not something Mozilla needs to be bothered with. Similarly, the current setting of '1' for network.sendRefererHeader should also likely be removed and treated as a 2. It's quite useless. 0 is also not very practical but recommended by the spec. It might be useful it if was on a prefbar type thing.

Same great taste, less confict with current code.

For those sites that are broken, would the possibility of sending only the host part of the referring URI (either for 3rd-party changes or all the time) work? Say Mozilla went to a page on www.mozilla.com from a Google JP search. With this option, Instead of either "lying" and sending http://www.mozilla.com/, or revealing everything or nothing, send http://www.google.co.jp/ WITHOUT the rest of the URI (therefore not revealing search terms or location within the site.)

> Say Mozilla went to a page on www.mozilla.com from a Google JP > search. With this option, Instead of either "lying" and sending > http://www.mozilla.com/, or revealing everything or nothing, > send http://www.google.co.jp/ WITHOUT the rest of the URI > (therefore not revealing search terms or location within the site.) Sounds like a good option to me...I'll whip up a new patch forthwith.

I've added the option to strip the referrer's path for 3rd party requests: // 6 - Send the actual referrer to all hosts, but strip off the path for 3rd party requests.

Excellent...SetReferrer() is getting closer and closer to my changes! :)

For example of what can be found in a referer header, see: <http://extremetracking.com/open;ref1?login=bob1647>. There are many local links/mail links that give paths which include user names. This opens up the layout of a person's profile on a computer so that a cracker knows exactly where to damage information. You can even using the same site, associate (with luck), the referrer with an IP address which increases the chances of a successful crack. I don't see any reason why referer headers even need sent from local files/mail.

brant: mozilla will only send http://, ftp://, gopher://, and in some special cases https:// URLs as HTTP referrers. there is an explicit whitelist.

Hi, a few comments: 1) Whatever bad karma comes from sending a deliberately false referer header probably also falls on anyone who sends a deliberately false User-Agent header (i.e. configures Mozilla to claim that it's MSIE). And yet it's often to the user's benefit to do this, so Mozilla provides a UI feature for the precise purpose of falsifying the User-Agent header. So those whose sensibilities are offended by falsifying Referer should also be arguing to remove the User-Agent preference UI. After all, look at all those browser popularity statistic gathering that gets messed up. Yet I think most people think being able to configure User-Agent is a good thing. 2. I don't understand comment #72. Merchants have no control over whether customer browsers send referer headers or not. Whatever agreement the payment processor has with the merchant is not really the customer's concern. If the customer's payment is declined, the customer will probably just buy the same thing from a different merchant. It's up to the merchant to slap some sense into the payment processor. Any payment processor who thinks referer headers are a "security" feature needs a smack of a clue-by-four anyway. 3. Mozilla 0.9.2 was obnoxiously aggressive about sending referer headers from non-web resources, including from bookmarks, from javascript, etc. I'm using 1.2.1 now but haven't tested if it's better. 4. One type of resource not mentioned in this thread is javascript. If I have a link on a page with href="javascript:window.open(some_url)", some_url should NOT receive a referer header and MSIE doesn't send one. Moz 0.9.2 sent a referer and I think that's a bug. 5. I agree with those who want the behavior of: send a referer if the referer is a web url on the same site as the target url. Otherwise, don't send one. I'm indifferent to whether there's a button to send fake referers, i.e. I don't especially want it but I don't oppose it and I disagree with the "bad karma" argument opposing it. 6. The really winning (but maybe complicated) fix for this is to allow running user-provided Javascript on every browser navigation, that can modify all the outgoing http headers as it chooses, sort of like MSIE Browser Helper Objects (BHO). That would allow combining all the UI issues of cookies, referer headers, user-agent strings, etc. into a single interface that XUL applications (like the xulplanet Preferences Toolbar) could operate. See my comment in bug # 79515 for a little more about this.

Oops, previous comment was supposed to refer to bug #75915 regarding a BHO-like inerface, not 79515 (typo). Sorry.

This is a security bug and there's a patch that fixes it. What is blocking this patch from being applied?

Duh. If I only remembered the new patch here in this bug. I recently merged my 1.0 patches with 1.4 and this bug was one of them (and the way hardest to merge, as always :-( , so could we please get the non-UI part in the tree?). for this reason, it bases on an older version (IIRC attachment from 2002-04-18), not the current one (attachment from 2002-11-20). However, I think the "current" one bitrotted again, so we now have 2 branches/forks of the patch *sigh*. Could we please get the fix for this bug (I don't care about which version, take the better one) into the tree, so save us from the mess? (oh, I said that already.) I leave it up to Jeff Thielke to sort out the differences/merges, sorry Jeff.

ops, left in some irrelevant Beonex changes. I only tested the "fake referrer to 3rd parties" pref, and that seems to work.

Same old patch, applies to the latest Mozilla. I obsoleted Ben's patch (id=127034), since it is based off an older version. This is the latest code, applies cleanly against Mozilla as of 07/07/03, and is fully tested. If someone were going to commit the referrer privacy feature, this is the patch you'd want...

what about a GUI for firebird?

Comment on attachment 127143 [details] [diff] [review] Patch applies cleanly as of 07/07/03 ask the Firebird people about a UI for Firebird.

I filed bug # 216596 asking for adding a GUI privacy option to change the referrer settings in firebird. The bug was confirmed and classified a future enhancement.

Is there any development with this bug or are all folks waiting for Darin Fisher's review/superreview?

The patch is done and working fine, tested by (tens of?) thousands of Beonex users and me since over a year. It's waiting for review, so that it can be checked in. Which is what the status whiteboard says, BTW.

Mike Tigas (aka MadmanNova) applied this patch into his MNG enabled nightly firebird builds. So from now on firebird users can use and test the patch. Testing by doing, rather than reviewing (wow, that rhymes ;) ). See the thread: http://forums.mozillazine.org/viewtopic.php?t=36797 Download here: http://www.missouri.edu/~msa0288/MozillaBuild/ These builds are built automatically on a daily basis. Works fine for me BTW. Thanks to Jeff and Ben.

@Darin Fisher: Is there any chance that there will be some comment from you regarding this patch? I think that the work that Ben Bucksch and Jeff Thieleke put into creating this patch deserve at least a r/sr- or a wontfix. In my eyes the current pure ignorance is rather disrespectful.

My appologies for not reviewing or providing much of any feedback on the patch. This bug has been in my future queue (low on my priority list), no one has pressed me to do anything with this patch, and I've been terribly swamped with other work :-/ That said, I will try to look at this patch during the 1.8 alpha development cycle.

Thanks Darin for the update. If you want a Win32 build to test the practical use of this patch, I can send you a 2004/01/22 firefox build that includes this patch (the last one that was built by Mike Tigas). Just send a mail to me.

(In reply to comment #116) > Thanks Darin for the update. > > If you want a Win32 build to test the practical use of this patch, I can send > you a 2004/01/22 firefox build that includes this patch (the last one that was > built by Mike Tigas). Just send a mail to me. > Do you happen to by chance have a 0.9 Firefox version? I'm interested in trying it out.

A 0.9 Firefox version with the patch manually applied can be found here: http://www.pryan.org/mozilla/firefox/amano/Fx-0_9-O1-Gopher-MNG-Exp.exe Go to this google site http://www.google.com/search?num=20&hl=de&ie=UTF-8&q=browser+referrer+test&btnG=Suche&lr= and choose the second link (asp.flaaten.dk/test/atest.htm). This is a test site that reports the referrer. It always reports should report google, unless you change the "referer" setting.

Works great! Thanks! Had to change network.http.sendRefererHeader to 6 before it worked like you said. Before that it was at 0. Are the settings as follows? * "Send no referrer to 3rd party servers" (network.http.sendRefererHeader = 4). * "Send a fake referrer to 3rd party servers" (network.http.sendRefererHeader = 5). * "Always send a fake referrer" (network.http.sendRefererHeader = 6). The last was later replaced with // 6 - Send the actual referrer to all hosts, but strip off the path for 3rd party requests. For what's it's worth, I'll be using 5 most of the time In the past I've always used proxomitron's default referrer blocking which I believe is closest to the old setting 6 (always fake) . It fakes it slightly different from your patch though I believe it sends back exactly the same referrer as the page being requested. So if you are visiting http://www.foo.com/example.html from http://www.test.com/index.html, it fakes the referrer as http://www.foo.com/example.html Not sure if it's any better or worse than sending http://www.foo.com as it is now. I don't think a UI is realistic to expect in Firefox for this, it seem to me faking referrers looks like one of those advanced options. I'm just happy for it to be in the core.

network.http.sendRefererHeader 5 does not work in Firefox 0.9. I set this using about:config, exited and restarted Firefox, went to http://www.google.com/search?num=20&hl=de&ie=UTF-8&q=browser+referrer+test&btnG=Suche&lr, and followed the 2nd hit and the full Google URI was sent, incorrectly, despite the setting. about:config shows it still set to 5, so I didn't somehow lose instead of saving the prefs change either. This, coming so soon after hearing that config.trimOnMinimize was added to the core and finding it also not working in the publicly available Firefox 0.9, is very disappointing. Are these fixes ever propagated to end-users that simply download the browser from the Web site and expect it to be reasonably up-to-date, or are they only ever available to people who use CVS repositories and other arcane stuff?

Neither this referrer patch nor the config.trimOnMinimize patch are included in the public 0.9 release. For the first, you have to stick to my manually patched builds, for the latter you have to use current trunk nightly builds.

This is ridiculous. Both of these bugs have been known for bloody years; both have patches that either fix the problem, or provide a workaround, and have had for months. Why is the publicly available browser not up to date with fixes developed months ago for bugs reported years ago? I thought it was Microsoft that was bad about taking ages to release fixes for bugs, but this takes the cake. I don't want to have to sign up for some sort of developer level access just to be able to browse with all of the fixes already made for bugs years old, and install the hundreds of megs of development environment needed to compile the sources. There's no point in fixing bugs if the fixes are not made available to the general public, but instead kept within some "inner circle" of developers for which you have to at least pay a steep price in disk space and download times, if not money or something. I wouldn't be so annoyed, but both these bugs have been in the database for years, and the patches for nearly as long. There's simply no excuse for them to be so slow in becoming available to end-users. Once a patch has been supplied that's tested and works it should be available as a bugfix update to the publicly available executables *immediately*. I want you to enact such a policy retroactively, which means I want to see an updated firefox 0.91 executable on your main download site within seven (7) days that is current as regards bugfixes for which patches have been submitted and tested and proven to work. Particularly for patches that have no effect if the user doesn't fiddle about in about:config there is no excuse for not bundling them into the very next public release. Right now, my feeling is that the publicly-available browser and, probably, other apps like thuderbird are lagging a full year, if not more, behind what's actually *tested and working*. If you're really worried about some obscure problem making a bugfix backfire then at least offer public betas that are well ahead of the current public releases! A year is an eternity in Internet time! Stop keeping all the yummiest bugfixes to yourselves and let the general public have them for crying out loud -- especially for showstoppers like the browser hanging when unminimized, and serious privacy holes like this bug attempts to address!

If you want a patch to be checked in, find someone to review it. Bugzilla is not an appropriate place to find a reviewer, because such people have often turned off their Bugzilla notifications. They do that so they won't have to wade through unhelpful and uninformed rants such as the comment immediately above. If you're not going to be part of the solution, please at least avoid being part of the problem.

The developers' internal affairs are not my concern -- I'm an end user. Having a product in front of me that works is my concern. Making that product, and making fixes available to the general public in a timely manner -- that is your concern. Also, four hours of your seven days are now up.

> Neither this referrer patch nor the config.trimOnMinimize patch are included in > the public 0.9 release. For the first, you have to stick to my manually patched > builds, for the latter you have to use current trunk nightly builds. Thanks for your excellent work. I was looking for 0.9.1. Until this bug is fixed, those of us who like this feature will have to depend on your generousity in helping. Thanks again,

As the author of this patch, I share many of derbyshire2@rogers.com's frustrations, although he/she is way off base with the rants, quasi-threats, and time limits. The facts are that this patch is substantially unchanged in over two years, it has been included in, and well tested by, several browser forks and private builds, and it has been reviewed and is simply awaiting Darin Fisher's superreview (see his April 5th post). Yet in two years, no one has seen fit to spend the 15 minutes it would take to review and commit this change, or otherwise make suggestions for improvement. At one point, I was very enthusiastic about Mozilla/Phoenix/Firebird/Firefox development. But basically due to situations like this, I have given up contributing patches to Bugzilla, and Mozilla development in general except for keeping my small set of patches up-to-date. I'm sure everyone is terribly heartbroken, but that is my story...

@Jeff write Darin a polite reminding mail, I'm sure he'll find the time then... Or ask him for someone else to review the patch, if he hasn't.

This is very interesting, because it's true. Mozilla development seems to have stopped, at least from my point of view Maybe Mozilla needs an automatic and democratic patch approval system and not just some dictators elected by nobody knows who, that just block user accepted patches for unknown reasons. Do you think this development model will scale in the future? Or will this development model kill mozilla?

(In reply to comment #128) > This is very interesting, because it's true. > Mozilla development seems to have stopped, at least from my point of view > > Maybe Mozilla needs an automatic and democratic patch approval system > and not just some dictators elected by nobody knows who, that just block > user accepted patches for unknown reasons. > > Do you think this development model will scale in the future? > Or will this development model kill mozilla? Wrong place to discuss this don't you think? If I didn't know better, I would have thought, people are trying to mess up this bug with irrelevant comments just to decrease it's chances of being approved.

Comment on attachment 127143 [details] [diff] [review] Patch applies cleanly as of 07/07/03 >Index: mozilla/netwerk/protocol/http/src/nsHttpChannel.cpp >+#define REFERRER_NONE 0 /* Never send the referrer */ >+#define REFERRER_USER_ACTION 1 /* Actions directly initiated by the user (e.g. clicking on a link) */ >+#define REFERRER_INLINE_CONTENT 2 /* Images or other inline content */ >+#define REFERRER_NON_HTTP 3 /* NOT USED - remains for backwards compatability */ >+#define REFERRER_SAME_HOST_ONLY 4 /* Send the referrer only for requests from the same host, otherwise send no referrer. */ >+#define REFERRER_3RDPARTY_PREPATH 5 /* Send the referrer only for requests from the same host, otherwise send target URI's pre-path as the referrer. */ >+#define REFERRER_3RDPARTY_NO_PREPATH 6 /* Strip off the path from the referrer for 3rd party requests, otherwise leave it alone. */ >+#define REFERRER_PREPATH_URI_ALWAYS 7 /* Always send the target URI's pre-path as the referrer. */ these shouldn't be lumped into one pref. what if i wanted both USER_ACTION and SAME_HOST_ONLY? this makes those mutually exclusive and that doesn't make any sense to me. perhaps these could be expressed as flags instead that can be bit-wise or'd together to select a referrer mode.

Should we make the pref handling deal with a comma-separated list of actual, meaningful strings, instead of bitflags? "user_action,same_host_only" Though we'd need to figure out if that sent the referrer in the union of those sets of events, or the intersection. (Paul Derbyshire, is that you?)

the pref value doesn't need to be user-friendly. i think we should minimize the footprint of this patch as much as possible... that is if it is to be included in necko... which, come to think of it... this feature could probably be implemented as an extension. we have nsIHttpChannel::referrer, which can be read and then set. why not listen for "http-on-modify-request" and then perform the referrer modifications there? (see the livehttpheaders extension to learn how to intercept this event.) then development of this feature would not be constrained by waiting for feedback from core mozilla developers such as myself. i personally don't see our default referrer policy changing if this patch were to land, so this patch would likely impact only a very limited number of mozilla users. i doubt firefox devs would accept UI for this feature... unless... sometimes it is easier to start with an extension, prove the concept, and then on the heals of that momentum, roll the feature into the default mozilla builds. or at least that's how it is supposed to work.

darin, I am using the patch in my browsers and have to merge it with new versions all the time, so I would appreciate it going into Mozilla directly.

> darin, I am using the patch in my browsers and have to merge it with new > versions all the time, so I would appreciate it going into Mozilla directly. But what about the browsers Mozilla.org ships? Those are my primary concern. Firefox has one answer with its extension system. If someone develops an improved patch for Necko, I will definitely consider it. I'm not saying this has to be done as an extension. I'm just saying that it is an option that would allow the developer(s) of this feature to move at their own pace. Moreover, doing this as an extension doesn't preclude it being added to Mozilla later on. In fact, if it is successful as an extension, then that would only help it find its way into the default Mozilla builds.

(In reply to comment #130) > (From update of attachment 127143 [details] [diff] [review]) > >Index: mozilla/netwerk/protocol/http/src/nsHttpChannel.cpp > > >+#define REFERRER_NONE 0 /* Never send the referrer */ > >+#define REFERRER_USER_ACTION 1 /* Actions directly initiated by the user (e.g. clicking on a link) */ > >+#define REFERRER_INLINE_CONTENT 2 /* Images or other inline content */ > >+#define REFERRER_NON_HTTP 3 /* NOT USED - remains for backwards compatability */ > >+#define REFERRER_SAME_HOST_ONLY 4 /* Send the referrer only for requests from the same host, otherwise send no referrer. */ > >+#define REFERRER_3RDPARTY_PREPATH 5 /* Send the referrer only for requests from the same host, otherwise send target URI's pre-path as the referrer. */ > >+#define REFERRER_3RDPARTY_NO_PREPATH 6 /* Strip off the path from the referrer for 3rd party requests, otherwise leave it alone. */ > >+#define REFERRER_PREPATH_URI_ALWAYS 7 /* Always send the target URI's pre-path as the referrer. */ > > these shouldn't be lumped into one pref. what if i wanted both USER_ACTION and > SAME_HOST_ONLY? this makes those mutually exclusive and that doesn't make any > sense to me. Values <= REFERRER_NON_HTTP are for backwards compatability only. And the only reason to differentiate between a user action and not (i.e. images) is because Mozilla is currently unable to do anything special with the referrer other than not send it. When you are able to control what referrer is sent, and more importantly, to which server/domain, there is no need to differentiate between a user and page initiated event. > i personally don't see our default referrer policy changing if this patch > were to land, so this patch would likely impact only a very limited number > of mozilla users. Why not? As was noted early in this bug, RFC 2616 (Section 15.1.2) notes that the current Mozilla referrer behavior can be dangerous. There are plenty of examples in this bug report as well. REFERRER_SAME_HOST_ONLY or REFERRER_3RDPARTY_NO_PREPATH, as the default referrer level, would both be appropriate for general use and then *all* Mozilla users would benefit from an increase in privacy. > sometimes it is easier to start with an extension, prove the concept, and > then on the heals of that momentum, roll the feature into the default > mozilla builds. or at least that's how it is supposed to work. There is no need to prove this concept - it has already been included in many Mozilla forks and private builds. It has been proven for going on two years now... Since the referrer header can be both a privacy and security issue, it needs to be part of the Mozilla core, just as surely as remote server image and cookie blocking should be part of Mozilla, and not some optional extension.

So long as this doesn't (by default) stop people who are browsing blogs from sending the full referrer URI when they click from one blog to another. It is imperative that we don't screw over the blogging community by breaking the extensive referrer tracking systems that have been set up to track discussions. Bloggers are amongst our most vocal advocates. Also note that most people don't care about this supposed "privacy leak", and some would go even further -- I, for example, actively _want_ to be sending referrer data. Don't try to "save" people by breaking their community!

(Weblogs generally gave up on referers years ago, because of pornographic referer spam, in favor of services like Technorati. Nevertheless, Mozilla probably shouldn't *default* to a more private setting until we learn how much of the Web it breaks.)

That would certainly break TONS of sites. Many, many servers (especially free hosting) use the REFERER to check for offsite linking of images (such a function is even described in the Apache documentation). A lot of online magazines also use it to tell if you've come from a search engine or their own website and render diferent content (perhaps based on their search keywords), for example. While the defaults might not break this directly, they most likely would for servers that have a list of "okay" REFERERs (or perhaps caches/multiple server front-ends) rather than using only the current domain name. These are just a few examples of how "most users" are going to be affected. If you want a broken Internet by default, that's fine, but don't make it broken for everyone by default.

> That would certainly break TONS of sites. Many, many servers (especially free > hosting) use the REFERER to check for offsite linking of images (such a function > is even described in the Apache documentation). A lot of online magazines also > use it to tell if you've come from a search engine or their own website and > render diferent content (perhaps based on their search keywords), for example. > While the defaults might not break this directly, they most likely would for > servers that have a list of "okay" REFERERs (or perhaps caches/multiple server > front-ends) rather than using only the current domain name. These are just a > few examples of how "most users" are going to be affected. If you want a broken > Internet by default, that's fine, but don't make it broken for everyone by default. The merits of referrer privacy have been discussed ad nauseam in this bug report, but let me just assert that I have been using this feature for over 2 years now, and over the course of that period, I have had almost no problems using REFERRER_3RDPARTY_PREPATH (*far* fewer problems than blocking 3rd party cookies). That said, I wouldn't have a problem with leaving the default value the way it is - just don't use the "we aren't going to change the default value so this patch isn't necessary" excuse.

this is not fixed. reopening bug.

The question of default should not be covered by this bug, but I think the HTTP spec alone (in addition to actual need) gives enough reason to say that this pref should be in Mozilla anyways, *at least* as backend pref. I need to merge this patch with Mozilla 1.7.x. Maybe I can find the time to adjust the patch to darin's demands, but the only point for me would be to get it into Mozilla, so I need to understand what exactly darin wants (I currently don't) and that he is generally OK with this.

What is needed is for someone to own the patch. I explained above that I do not think it is correct to overload the existing preference. I would recommend that someone interested in owning this patch address that issue first. I can't promise that I won't find other issues with this patch. Code development is an iterative process. If you per se want this patch to go into Mozilla, then step up to the plate and drive the patch.

Well, I think the default should simply stay the way it currently is(sending the correct referer). We should let the end-user decide if they want to change anything and if so, in what way. As this will be a presumably hidden preference for the more 'advanced' users, I dont see this as a problem. As a result, all the preferences mentioned above should be available as a precautionary measure.

i read through the comments and ordered my impressions by listing the made suggestions in a flag list as proposed by darin (about the comment #130 .. the missed option would read like ar:always;er:same_host) didnt know where to place the cut_away_cgi_arguments .. about the handling of the feature i think it should be only visible in the prefs aka about:config .. as ex with the trivial cases 0-7 and an 8 for the flaglist or a complete flaglist switch .. or as said a recoding as a .js component though wondering myself how to elegantly distinguish ar from er .. ------------ action referer .. (ar) // if user clicks on a link never .. same_host .. always inline/embedded content referer .. (er) never .. same_host .. always // fake and blacklist referer behaviour are if enabled the counterparts of ar and er like: // always .. not same site .. never // else there would imho occur logical collisions fake referer .. (fr) never .. ar .. er .. ar&er blacklist referer .. (br) // overrides fr .. element impl as comma sep list ? as ex for search sites disable .. ar .. er .. ar&er blacklist referer send behaviour .. (brsb) send_no_referer .. cut_away_cgi_arguments .. send_fake_referer

More than 10 years later, part of this were implemented in bug 822869. Thanks!

I think as close as we're going to get was implemented in comment 146 and meta referrer (other bug)