From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.14 i686; en-US; m17) Gecko/20000717 BuildID: 2000071708 Both this build and the latest milestone M16 will not work with Microsoft Proxy server ver2.0. after configuring the proxing settings in Mozzilla prefs, the browser will not display any pages. Reproducible: Always Steps to Reproduce: 1.Change proxy setting to point to the correct port on a MS Proxy Server. 2.Attempt to browse the web 3. Actual Results: Unable to open any web page. Browser does not make any attempt to authenticate with the proxy Expected Results: Worked like IE, and prompted for a Username/Password, then allowed browsing if password is correct n/a

*** This bug has been confirmed by popular vote. ***

Verified for M17 on Win95 via MS Proxy 2.0. I don't believe this is related to the password dialog/authentication persistance bugs; no password dialog appears at all, no page is rendered, and the URL in the toolbar doesn't change if you use a bookmark. But the status bar says "Document: Done", so that's nice. :)

MS Proxy 2.0 has problems with HTTP/1.1 Ruslan you wanna investigate further?

Not surprisingly, this bug also exists on a Mac using 2000081620. Setting HTTP= 1.0 in the debug preference makes no difference. This bug should be nominated for nsbeta3 since all too many corporate sites use Microsoft Proxy 2.0.

nominating nsbeta3

triage team [NEED INFO] Are there later versions of Microsoft Proxy server, and do we work on the later versions?

[INFO] Proxy Server 2.0 is the newest version. additional information can be found at http://www.microsoft.com/proxy/default.asp

[INFO] MS Proxy Server is being superceeded by MS ISA Server 2000 for Win2K (Internet Security and Acceleration !) currently beta - this also uses application proxies for http, https and ftp like MSPS2... See http://www.microsoft.com/isaserver/

[NOTE TO STEVE] Unfortunatly, many companies have large numbers of MS proxy servers running on the NT4.0 platform. Proforming a major upgrade of operating systems, and the swiching from proxy to ICA is a major project. If the systems is running fine, an orginization will not give funding just to make the system compatible(mabey) with mozilla. Jon

[NOTE TO JON] Sorry, after re-reading my post was a little off-thread and I meant that we have to be aware if the problem also exists for MS-ISA as well as MS-PS2. Also, any solution should resolve for both proxies/firewalls if the problem is present with both.

This bug is actually a regression. I just checked and M15 works fine with Microsoft Proxy Server 2.0, but fails for M16, M17, M18, and currrent nightly on MacOS.

Maybe we need a preference that says use HTTP/1.1 with proxies or not. BTW this is not a regression since in M15 we weren't fully HTTP/1.1 compliant. Ruslan, you think that would be doable?

Proxies are generally seem to be busted now for some reason. Setting http.version=1.0 will also affect proxies (proxy-version is set to be the same as normal http version). I think it's not related to http version in this case.

We need a new preference that just switches HTTP/1.1 off for proxies.

Someone please change the OS for this bug to All. Also, IE has a "Use HTTP 1.1 through proxy" option under it's advanced settings. At our site IE routinely crashes when you enable this. Please don't underestimate the severity of this bug; this will be preventing large numbers of corporate users from using Mozilla outside of their own intranets.

We can have a preference - but somebodys will need to add it to prefs dialog as well, nicely.

Fix at hand. Gagan - can you review/approve it etc. ....?

Not holding PR3 for this; marking nsbeta3-. Please nominate for rtm if we really need to fix this before shipping Seamonkey.

Can you reconsider this? I think it's important enough to get into the final Seamonkey version, otherwise you'd be excluding a significant number of (most likely) corporate users. Just my two cents...

sorry for the delay, i've been out of town since Wednesday. sr=mscott

Any chance that Ruslan's patch could be checked into the trunk? This has been broken since M15. It would be nice to test a recent build.

ruslan: any chance you can check this into the trunk? You have my approval. Thanks!

no prob. Will try to do it tomorrow.

If changes have been committed already, they have not yet fixed the problem. I tried applying the patch by hand as well to existing source tarball, but that did not seem to work either.

What do you mean it doesn't work? Did you set proxy http version pref to be 1.0?

[TO ruslan]: With the latest nightly build (today's), the proxy is set as it is in regular netscape 4.7, and the http version (under Debug) is set to 1.0. The problem is the same as it has been. No prompt for authentication, document says its done loading, but shows in the main window. Loading local documents or intranet documents works fine (setting the proxy to ignore 166.86.7.). When I tried putting the changes in by hand with a download of the source, I have the same problem. It is possible I was not correctly incorporating the change.

sorry, that should be "but NOTHING shows in the main window"

Ok, this bug is fairly important for us. We are hoping to move our thin clients from using netscape 4 to netscape 6 or mozilla. However, all of our thin clients go through MS Proxy II. So all of our testing is blocked at this point in time. Here is some info that may help you out. I figured out what the browsers were actually sending by writing a miniture server (most of the code comes from a textbook though, so I can't take credit). The server listens to the http port (80) on the localhost, and echos to the screen whatever it receives (I'll try to attach it as a tgz in just a little while). Where I say "proxy GET", I mean I set the browser's proxy to localhost:80, and tried to contact http://www.mozilla.org. Where I say "regular GET", I mean I set the browser to use no proxy and tried to load http://localhost/. Here are the requests that the browsers are sending to the server. The first two proxy requests(mozilla) do not work, but the third(4.75) does: ################################################################ This is a nightly build that let's me set the HTTP version under "Debug". Proxy GET in /pub/mozilla/nightly/2000-10-19-21-Mtrunk: GET http://www.mozilla.org/ HTTP/1.0 If-Modified-Since: Sat, 21 Oct 2000 08:11:28 GMT If-None-Match: "6a611-2bb5-39f14fb0" Host: www.mozilla.org User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.16-22 i686; en-US; m18) Gecko/20001019 Pragma: no-cache Cache-Control: max-age=0 Accept: */* Accept-Language: en Accept-Encoding: gzip,deflate,compress,identity Keep-Alive: 300 Connection: keep-alive Regular GET in /pub/mozilla/nightly/2000-10-19-21-Mtrunk: GET / HTTP/1.0 Host: localhost User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.16-22 i686; en-US; m18) Gecko/20001019 Accept: */* Accept-Language: en Accept-Encoding: gzip,deflate,compress,identity Keep-Alive: 300 Connection: keep-alive ################################################################ The latest nightly no longer has the "Debug" menu. Proxy GET in Mozilla(Nightly Oct 21 2000): GET http://www.mozilla.org/mozorg.html HTTP/1.1 If-Modified-Since: Thu, 27 Jul 2000 05:19:14 GMT If-None-Match: "7315f-17ab-397fc652" Host: www.mozilla.org User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.16-22 i686; en-US; m18) Gecko/20001021 Pragma: no-cache Cache-Control: max-age=0 Accept: */* Accept-Language: en Accept-Encoding: gzip,deflate,compress,identity Keep-Alive: 300 Connection: keep-alive Regular GET in Mozilla(Nightly Oct 21 2000): GET / HTTP/1.1 Host: localhost User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.16-22 i686; en-US; m18) Gecko/20001021 Accept: */* Accept-Language: en Accept-Encoding: gzip,deflate,compress,identity Keep-Alive: 300 Connection: keep-alive ################################################################## Regular Netscape works fine. Proxy GET in Netscape 4.75: GET http://www.mozilla.org/ HTTP/1.0 If-Modified-Since: Sat, 21 Oct 2000 08:11:28 GMT; length=11189 Proxy-Connection: Keep-Alive User-Agent: Mozilla/4.75 [en] (X11; U; Linux 2.2.16-22 i686) Pragma: no-cache Host: www.mozilla.org Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */* Accept-Encoding: gzip Accept-Language: en Accept-Charset: iso-8859-1,*,utf-8 Regular GET in Netscape 4.75: GET / HTTP/1.0 Connection: Keep-Alive User-Agent: Mozilla/4.75 [en] (X11; U; Linux 2.2.16-22 i686) Host: localhost Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */* Accept-Encoding: gzip Accept-Language: en Accept-Charset: iso-8859-1,*,utf-8

This bug should be marked as 'All' for OS. Using Brandon's server with M18 for Win95 gives: GET http://www.mozilla.org/ HTTP/1.0 Host: www.mozilla.org User-Agent: Mozilla/5.0 (Windows; U; Win95; en-US; m18) Gecko/20001010 Accept: */* Accept-Language: en Accept-Encoding: gzip,deflate,compress,identity Keep-Alive: 300 Connection: keep-alive IE 4.01 (which works with MS Proxy 2 amoung others) gives: GET http://www.mozilla.org/ HTTP/1.0 Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* Accept-Language: en-gb Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 4.01; Windows 95) Host: www.mozilla.org Proxy-Connection: Keep-Alive It looks like the 'Connection' header needs to be replaced with 'Proxy- Connection'. It also seems that all the MS Proxy 2 friendly browsers, including Netscape 4.75, avoid the 'Keep-Alive' header too. Interestingly, IE 4.01 sends out exactly the same headers (apart from the HTTP version number) if you enable the 'use HTTP/1.1 through proxies' option. I can post/upload more examples if it would help.

As suggested, changing OS to all (I do remember seeing this when we had MS Proxy).

Cool research. I think you're right about proxy-connection. The fix should be fairly simple.

Ok, got an updated version. An example run would be: ./proxy msproxy.ccs.com 8765 80 where proxy is the executable name, msproxy.ccs.com is the name of the offending real proxy, 8765 is the port to listen on the local machine, and 80 is the port of the real proxy. After starting ./proxy, set up netscape to use localhost:8765 as the http proxy. Using the updated version, I think the problem may be that Mozilla just does not handle proxy authentication: This is a session with regular netscape 4.75 loading a simple file I put on my old student account. Note, where it says "session complete", that comes from my program, to indicate that the connection was closed.: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% GET http://www.cslab.vt.edu/~rostewa2/ HTTP/1.0 If-Modified-Since: Mon, 23 Oct 2000 22:42:44 GMT; length=14 Proxy-Connection: Keep-Alive User-Agent: Mozilla/4.75 [en] (X11; U; Linux 2.2.16-22 i686) Pragma: no-cache Host: www.cslab.vt.edu Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */* Accept-Encoding: gzip Accept-Language: en Accept-Charset: iso-8859-1,*,utf-8 HTTP/1.1 407 Proxy authentication required Proxy-Authenticate: NTLM Proxy-Authenticate: Basic realm="192.168.11.5" Content-Length: 503 Content-Type: text/html <html><head><title>Error 407</title> <meta name="robots" content="noindex"> <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1"></head> <body> <h2>HTTP Error 407</h2> <p><strong>407 Proxy Authentication Required</strong></p> <p>You must authenticate with a proxy server before this request can be serviced. Please log on to your proxy server, and then try again.</p> <p>Please contact the Web server's administrator if this problem persists.</p> </body></html> session complete ########################################### The session's not complete. At this point netscape prompts me for a name and password. After I enter those in, the session continues. ########################################### GET http://www.cslab.vt.edu/~rostewa2/ HTTP/1.0 Proxy-Connection: Keep-Alive User-Agent: Mozilla/4.75 [en] (X11; U; Linux 2.2.16-22 i686) Pragma: no-cache Host: www.cslab.vt.edu Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */* Accept-Encoding: gzip Accept-Language: en Accept-Charset: iso-8859-1,*,utf-8 Proxy-authorization: Basic c3Rld2FydGI6b21keXlqcnkubQ== HTTP/1.1 200 OK Via: 1.1 DONCARLUCCI Proxy-Connection: Keep-Alive Content-Length: 14 Content-Type: text/html Date: Tue, 24 Oct 2000 17:33:00 GMT Server: Apache/1.3.12 (FreeBSD) PHP/3.0.16 mod_ssl/2.6.4 OpenSSL/0.9.4 Last-Modified: Mon, 23 Oct 2000 22:42:44 GMT ETag: "38b26-e-39f4bee4" Accept-Ranges: bytes Keep-Alive: timeout=15, max=100 <html></html> session complete %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Ok, now we do it again with mozilla. GET http://www.cslab.vt.edu/~rostewa2 HTTP/1.0 Host: www.cslab.vt.edu User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.16-22 i686; en-US; m18) Gecko/20001023 Accept: */* Accept-Language: en Accept-Encoding: gzip,deflate,compress,identity Keep-Alive: 300 Connection: keep-alive HTTP/1.1 407 Proxy authentication required Proxy-Authenticate: NTLM Proxy-Authenticate: Basic realm="192.168.11.5" Content-Length: 503 Content-Type: text/html <html><head><title>Error 407</title> <meta name="robots" content="noindex"> <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1"></head> <body> <h2>HTTP Error 407</h2> <p><strong>407 Proxy Authentication Required</strong></p> <p>You must authenticate with a proxy server before this request can be serviced. Please log on to your proxy server, and then try again.</p> <p>Please contact the Web server's administrator if this problem persists.</p> </body></html>session complete %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Mozilla just stops. It does not seem to recognize that it needs to get authentication information from the user and send it to the proxy. But more interestingly, it displays nothing on the window. So the user does not even get the benefit of knowing that proxy authentication is required. So mozilla seems to detect the need for authentication, but just isn't doing about it. Maybe Andrew can do some more detective work and find out more interesting things.

If somebody want to get me either code or a precompiled binary of this latest attachment (10/24/00 10:49) for Win32 (or tell me how to modify it to work) I'd be happy to gather some data from a working situation. I have a Gauntlet Firewall that reguires authentication and does work great with Mozilla (with the possible exeption of bug 57159). I have a copy of VC, but unfortunately am a little lacking in C skills.

Anyone out there able to help jake out? I don't have windows development tools to port the code myself. Besides, I don't know much about windows sockets. The code is fairly simple, so it shouldn't take much.

There are two problems here: 1) The code has regressed because it does not respond to proxy authentication requests, i.e. 407's. This seems to have been around for a while, see bug 22405 which is probably incorrectly closed. 2) There may or may not be a problem with setting HTTP/1.0 for proxy connections as I speculated above regarding 'Connection' and 'Proxy-Connection'. I can confirm Brandon's findings using his proxy-proxy on an IRIX box and pointing M18 on Win95 at it. Here's the log: GET http://www.purple.com/ HTTP/1.0 Host: www.purple.com User-Agent: Mozilla/5.0 (Windows; U; Win95; en-US; m18) Gecko/20001010 Accept: */* Accept-Language: en Accept-Encoding: gzip,deflate,compress,identity Keep-Alive: 300 Connection: keep-alive HTTP/1.1 407 Proxy authentication required Proxy-Authenticate: NTLM Proxy-Authenticate: Basic realm="www.purple.com" Content-Length: 503 Content-Type: text/html <html><head><title>Error 407</title> <meta name="robots" content="noindex"> <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1"></head> <body> <h2>HTTP Error 407</h2> <p><strong>407 Proxy Authentication Required</strong></p> <p>You must authenticate with a proxy server before this request can be serviced. Please log on to your proxy server, and then try again.</p> <p>Please contact the Web server's administrator if this problem persists.</p> </body></html>session complete At this point nothing more happens; there should be a login/pasword dialog displayed. This used to happen (many milestones back) but even then I think there was a problem communicating with MS Proxy 2.0. I just want to ram home the point that proxying is probably broken in *multiple* places. Anyway, thanks for the code Brandon. I've been wanting to do this for a while but didn't have the knowledge/time/books for it. Apologies, but I can't help the Windows people port the proxy-proxy code.

Rather than wait for someone to help me port the C code to windows, I did up some VB Code that does almost the same thing. I attached that and my results from a working situation (my Gauntlet firewall required Authentication and works fine with Moz). Interestingly enough, my code works fine with Moz and NS, but fails with IE.

Jake, could you post the transcripts of Mozilla talking to your proxy?

It's in the second attachment... http://bugzilla.mozilla.org/showattachment.cgi?attach_id=17962 I guess I should've mentioned in my description that it was a transcript...

Adding RELNOTErtm to keywords.

I know exactly what the problem is now. HURRAY!!! I actually am posting this from the latest nightly of mozilla from behind a msproxy II proxy. So everyone rejoice. If we look closely at what the difference is between the response that Jake's friendly firewall gives when authentication is required, i.e.: HTTP/1.0 407 Proxy authentication required Proxy-Authenticate: Basic realm="" and the response that MSProxy II gives, i.e.: HTTP/1.1 407 Proxy authentication required Proxy-Authenticate: NTLM Proxy-Authenticate: Basic realm="192.168.11.5" Content-Length: 503 Content-Type: text/html <html> . . . </html> We see that MSProxy is giving TWO authenticate lines. Now, I don't know what NTLM is, but when I modified my proxy to excise that line, everything just started working. So I am attaching the modified proxy, which is quite a hack, I apologize, but it works for me. Mozilla ends up seeing, instead of the above transcript, something like HTTP/1.1 407 Proxy authentication required Proxy-Authenticate: Basic realm="192.168.11.5" Content-Length: 503 Content-Type: text/html <html> . . . </html> To use the hack, run something like ./proxy msproxy.ccs.com 765 80 from the command line, where msproxy.ccs.com is the MS proxy you must go through 765 is what ./proxy is listening to for connections 80 is the port that your MS proxy uses Then set your proxy as localhost:765 I make no gaurantee that this will work on your machine, but it works for me.

Contgrats... Now, here's the part many will love... You can make MS Proxy server do this without using a modified localhost proxy! Basically what's happening is MSProxy is telling you it supports two types of Authentication: Basic and NTLM (a propritary MS Auth method, see bug 23679). All you have to do is config MSProxy to *not* accept NTLM. I can post rough instructions for doing this, but being that we've uninstalled MSProxy Server, it's only from memory. Open MMC (MS Managment Console) and go to the properties for the Proxy Server. I believe you'll find a security tab. Either on that tab or a using another command button there should be options for Anonymous, Basic, and Windows NT Challenge/Responce... you'll want to uncheck the NT C/R. Please notes, this is not a fix, just a work-around. Until such time as bug 23679 is fixed, mozilla should ignore the "Proxy-Authenticate: NTLM" line (in fact, it should igonore any Proxy-Authentication it doesn't support, esp if it finds one it does).

I can confirm that the incoming "Proxy-Authenticate: NTLM" header is causing the problem. I'm using Brandon's filtering proxy relay and M18 for Win95; I'm posting this comment using it. Nice one Brandon! Mozilla would become my default browser now, if the proxy wasn't leaving me with one zombie process per connection. Oh well, I'll see if I can fix it.

Evidently a similar bug existed for multiple WWW-Authenticate headers (bug 44041). Perhaps the patch for 44041 should be factored into a new routine that would handle both WWW-Authenticate and Proxy-Authenticate headers?

*** Bug 49931 has been marked as a duplicate of this bug. ***

justin, you did the fix in bug 44041. Do you think it's possible to the same here?

no fix in sight.

I would fix this myslef, but I don't know C/C++... in fact, I was on http://lxr.mozilla.org earlier today looking through the source code to see if I could make the fix from bug 44041 work on this, but discovered I don't even know enough about C to figure out where to put the code, let alone actually do it! Being that it looks like this will miss RTM, adding mozilla0.9 keyword so it will hopefully be fixed soon after.

The fix should be easy actually. It's marked rtm- and there's a workaround, so I don't know how important it is anymore. I'll fix it on the tip when I get some free chunk of time.

The patch for bug 44041 fixes both problems already, for the most part. But it only works for multiheaders which have been separated by LFs (linefeeds, I guess). The patch I'm posting separates multi-proxy-authenticate headers by linefeeds as well, allowing the patch for 44041 to work for both.

fwiw, from the resource kit NTLM authentication protocol A challenge/response authentication protocol. The NTLM authentication protocol was the default for network authentication in Windows NT version 4.0 and earlier. The protocol continues to be supported in Windows 2000 but no longer is the default. from http://www.microsoft.com/technet/network/distsec.asp NTLM=NT LAN Manager from search.microsoft.com there's mention of problesm w/ ie mac's support for it, so that means you don't need to be on windows to try to support it. Talk to the samba people if you care. </offtopic>

*** Bug 59485 has been marked as a duplicate of this bug. ***

*** Bug 59720 has been marked as a duplicate of this bug. ***

Gagan: How about a review for the 11/3 patch? If this patch solves the MS Proxy 2 problem, then perhaps another bug could be opened to track ruslan's patch for HTTP 1.0 support?

neeti/darin should track/review this now.

The 11/3 patch looks good to me. r=darin

Just one question... Is everyone in agreement that these proxy problems do not require HTTP/1.0 as part of the solution?? If so, then I'm fine moving ruslan's patch to another bug report.

I'm pretty sure that MS Proxy 2 supports HTTP/1.1 but I can't test it to be sure. According to Brandon's comment about his 10/26 attachment, Mozilla works fine if the "Proxy-Authenticate: NTLM" header is removed. This says to me that the 11/3 patch will solve the problem this bug describes and the HTTP/1.0 is another issue.

Sounds good... thx.

sr=mscott for the 11/03 patch.

darin: This has a R= and a sr=, can it be checked in now?

Yes... the patch can go in just as soon as the trunk is opened. If no one claims responsibility for landing this patch by the time the trunk is opened, then I'll just take care of it.

OK, I landed the 11/03 patch.

I pulled out the HTTP/1.0 proxy pref as bug 60811.

clarified summary to be consistent w/ other NTLM and auth header bugs I have cleaned up.

+qawanted - MS Proxy 2.0 is probably my next test server deployment...

*** Bug 47591 has been marked as a duplicate of this bug. ***

-relnoteRTM - NTLM still needs to be implemented, and that will be relnote in the implementing bug.

VERIFIED/fixed: if it wasn't working, we wouldn't have so many bug reports asking for NTLM for proxy and web.