At my site all https requests must go through a proxy server, but there's currently no way to set the name of your https proxy server in the UI. If I set the http (no s) proxy server to the correct value (at my site the http and https proxy servers are the same) and try to visit any https page, the browser times out after about a minute and doesn't display the https page.

i think this is a dupe of 31164. i will add your comments to that report. *** This bug has been marked as a duplicate of 31164 ***

not a dupe. This bug (31174) is https doesn't work through proxy - not psm window can't open because of proxy. I have a proxy and i can still view https sites through it. I just can't change my psm preferences in that new window that pops up when you click on the lock icon - (which I filed as bug 31164)

The PSM SDK now provides API to create a secure proxy connection. The secure proxy connection should go through PSM to succeed. There is a function that opens a connection to a secure proxy server (note that it starts out as a cleartext connection), and the other that "turns on" security on the socket. The protocol handler should be able to use these functions to implement the secure proxy.

So, who's the owner of the https protocol handler?

fyi I can duplicate the bug with 2000031412 on Win98. I did a sniffer trace and it definately is not sending https: data through the proxy (nor DNS lookups for https connections). -me

CCing some people, who might know, who owns the https code.

Has anyone had any luck tracking this one down? -me

Bob, is this really your bug? If not, can you assign to the correct developer, please?

taking over...

Bug 36125 filed about prefs UI.

there were really 3 bugs here... 1. the pref for HTTPS is not being set. 2. the pref is not being conveyed to the channel from the proxy protocol service. 3. the proxy setting conveyed to the https channel is not being used. I have fixed the #2. BenB is filing a new one for #1 and I am assigning this back to lord for #3.

This bug appears also for snews. Updating SUMMARY. Seth Spitzer, do you need to make additional changes similar to #2 for snews?

Raising priority - This bug prevents at least 15% of Mozilla users from testing PSM/NSS.

https is mine.

Reassigning all https/cartman/security bugs to valeski. He will be finding new owner(s). This shift is so that I can focus on embedding issues. If the new owner has questions that can not be resovled, I may be able to lend a (quick) hand. over to valeski....

Changing QA contact.

*** Bug 43141 has been marked as a duplicate of this bug. ***

Part #3 of this bug (actually using the proxy setting) is near code I've recently been changing due to socks stuff. I've enabled the PSM proxy code in the SSL IO layer, so I wouldn't be surprised if adding the code to nsProtocolProxyService is all you need to do. The proxy settings should get through to the io layer, which should do the right thing with them. I'll look over it again, just to make sure there's nothing left out. Also, I can add the UI and config code in about 10 minutes if you want me to take care of this bug entirely.

Assigning to gagan per PDT.

justin: go ahead. thx.

From what I can tell, the transport gets and uses the proxy info (part #3) I think the problem people are seeing here is that mozilla is trying to talk in SSL to the proxy server. And based on the description of the reporter's set-up, the proxy server doesn't speak in SSL. If that's the case, then this is really a duplicate of bug 40203. We can do HTTP proxy with SSL in two ways: 1) mozilla speaks SSL to the proxy, and does a normal GET http://foo/ proxy request 2) mozilla speaks cleartext with the proxy, does a CONNECT foo:port, and then starts up SSL and writes the request to the dest host directly (which is what bug 40203 implements) I can't imagine we'll see number 1 often, so how about we always take path 2. And if we do so, what's the purpose in a separate SSL proxy server configuration?

Thanks for your help Justin. ruslan/dougt: do you guys have more info on this CONNECT call question from justin? Can me make it always be that way? Seems like an ok thing to do but I don't know enough about this to make that call.

*** This bug has been marked as a duplicate of 40203 ***

Verified duplicate.

I updated, installed PSM, restarted Mozilla, verified that the Security Manager shows up correctly, verified that the proxy is correctly shown in Prefs|Advanced|Proxies|SSL and stored in prefs.js ( user_pref("network.proxy.ssl", "myproxy"); user_pref("network.proxy.ssl_port", 1111); ) and tried to load <https://www.fortify.net/sslcheck.html> (and other SSL pages), but I get * Unsupported Request Method and Protocol Squid does not support all request methods for all access protocols. For example, you can not POST a Gopher request. Works fine with 4.x. REOPENing.

I believe you need to set one more pref. It's called netwerk.*.ssl.connect or smth like that. Please retest with it.

What is it for? Anyway, neither true nor false work.

it's called "network.http.proxy.ssl.connect", default false BTW.

Per gagan, moving from [nsbeta2+] to [nsbeta2-]

I tried to set "network.http.proxy.ssl.connect" to true on 2000072809 and still got "GET https://..." delivered in clear text to my proxy (squid). Second. How much proxies do speak in SSL? I know no such (because technically it is [near to] imposible to create [good] one) , while CONNECT understand most. So, please, make use CONNECT default if https proxy setting is set.

The estimation in the status whiteboard is propbaly wrong now. Could somebody please tell, what I'm supposed to do in order to create a SSL connection to a webserver over a usual proxy (e.g. squid)? I should work, right?

As per http://www.alternic.org/drafts/drafts-l-m/draft-luotonen-web-proxy-tunneling-01.txt provided with bug 40203 program hould 1) establish tcp/ip connection to https port; 2) send "CONNECT hostname:portnum HTTP/1.0\n" 3) optionally send "User-agent: bla bla bla\n" 4) optionally send "Proxy-authorization: basic hidenPasswd\n" 5) send "\n" 6) receive HTTP responce header "HTTP/1.0 200 Connection established\nProxy-agent: squid/1.23\n\n" all until empty line 7) on that connection speak SSL as like just established direct connection with SSL server. I just checked with my squid - this scenario works. Have a good look!

*** Bug 46627 has been marked as a duplicate of this bug. ***

oops. meant to add-- considering the severity of this problem now (as compared to its original form) I am marking this as nsbeta3-

OK, finally started tcpdump. Mozilla seems to issue a GET, not CONNECT: GET https://www.trustcenter.de/ HTTP/1.1 Host: www.trustcenter.de User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.16-3 i686; en-US; m18) Accept: */* Accept-Language: de, en Accept-Encoding: gzip,deflate,compress,identity Keep-Alive: 300 Connection: keep-alive 4.x does a CONNECT and works.

Confirmed this, my firewall only understands CONNECT for the SSL proxy connections.

I'm using Squid 2.1. kaspar, which proxy do you use?

I should issue connect. Did you turn on all the prefs? It's still on my list to clean up confusion out of preferences.

squid-2.3.STABLE4

The same problem if proxy server is Apache-1.3.12

Adding myself to CC list; SSL via proxies is vital to Sun and other customers who are behind firewalls.

ruslan, see my comments at 2000-07-22 and the day after.

Doesn't work with Winproxy 2.1 either

qawanted. I'd like to know what is going on here. SSL through proxy has to work.

Tested again on Windows NT build 2000081308 and any secure connection crashes the browser totally. The one from the 08 Aug just failed to connect. The depends on bug id #40203 is marked as fixed, and this bug appears to be becuase of that problem. Is it?

*** Bug 46810 has been marked as a duplicate of this bug. ***

Tom? Where's my setup?

*** Bug 49869 has been marked as a duplicate of this bug. ***

OK, we have 5 people for whom this is not working, inclusing one crash. qawanted has been ignored so far. Removing nsbeta3- for reevaluation and adding dogfood.

actually, this is not really dogfood for me /right now/. But it is an essential feature that has to make it in final, and dogfood is the only way to get pdt attention these days, i think.

It's dogfood for *me*, and the rest of Sun. We are behind a firewall, and our only access to secure sites outside of the Sun firewall is via proxy servers. We cannot currently use Netscape6 or Mozilla as our default browser at Sun until this is fixed. Our CEO would like to switch to Netscape 6 as the default browser used inside of Sun today (like, really: today). But until this bug is fixed, we can't do it. I'm happy to go on record as nominating it as dogfood.

Putting on dogfood+ radar.

Ok. Our proxy support appears to be totally busted. I can't even get to the place where it's supposed to do CONNECT handshake. We've a major regression - it never even seems to set overriding spec on the url. Very bad. This bug can only be fixed after general proxy bustage is fixed.

Concerning the general proxy bustage, what happens at line 1474 of netwerk/protocol/http/src/nsHTTPChannel.cpp? Is SetOverrideRequestSpec being called? If not, which of the tests is false?

No. It is not being called.

Setting to P1.

I don't understand the state of this bug. To me it seems the SSL through proxy is working. (Linux nightly 2000090621 with PSM 1.2)

Worksforme with the 9/8 Linux and Win98 builds, and the 9/7 Mac build.

Verified worksforme.

I cannot do any kind of surfing(ssl/non-ssl) when the proxy is set on. I am using a proxy server up here at santa clara http://troll.red.iplanet.com/red-aol.proxy. John : can you give me a server I can use as proxy to test this.

SSL through a Proxy is not working for me with build 2000090704 (Windows) and PSM 1.2. When I try to connect to https://www.verisign.com, I get a couple of page faults in the PSM: PSM caused an invalid page fault in module NSPR4.DLL at 0137:3000295b. Registers: EAX=00906ae0 CS=0137 EIP=3000295b EFLGS=00010206 EBX=817d7c00 SS=013f ESP=00fefec8 EBP=00fefefc ECX=04833300 DS=013f ESI=00908840 FS=5b37 EDX=00feff3c ES=013f EDI=009088e8 GS=0000 Bytes at CS:EIP: ff 51 44 83 c4 14 c3 90 90 90 90 90 90 90 90 90 Stack dump: 00906ae0 00feff3c 00000008 00000000 ffffffff 0040e2fc 00906ae0 00feff3c 00000008 00000000 ffffffff 00000000 00000000 00feff48 00407a6f 00906ae0 PSM caused an invalid page fault in module <unknown> at 00de:009046f9. Registers: EAX=00906ae0 CS=0137 EIP=009046f9 EFLGS=00010206 EBX=817c9fa8 SS=013f ESP=0110fed8 EBP=0110ff10 ECX=009046ec DS=013f ESI=009085c0 FS=5b47 EDX=0110ff28 ES=013f EDI=00908668 GS=55ae Bytes at CS:EIP: 87 90 00 b0 83 90 00 ec 19 79 00 ec 19 79 00 20 Stack dump: 3000298e 00906ae0 0110ff28 00000008 00000000 ffffffff 0040e360 00906ae0 0110ff28 00000008 00000000 ffffffff 00404ad9 00000000 0110ff48 00404b77 After clicking "close" on the faults, the firewall returns the following error page: > URL form not supported > The URL that you tried to use is not supported by the proxy. This may be due > to an unsupported scheme, or possibly that the URL was malformed. The URL that > was seen by the firewall was: > > https://www.verisign.com/ > If this is an error, then you should contact your local firewall > administrator. Our proxy (Raptor-Eagle 4.0) requires use of the CONNECT method on port 80 to establish a tunnel to the remote server.

Nitin, try w3proxy.netscape.com:443. Also remember that you cannnot reach internal web sites then, so try https://www.verisign.com

junruh, Did your resolution "WORKSFORME" come from using proxy on 443 port which most likely _KNOWS_ how to speak in SSL? If that's the case, could you please try again with squid/apache/etc what understand only CONNECT method. And if it will work, could you please document _HOW_ to you achieved that. Thanks in advance.

Does not work for me with build from Sun with Squid proxy. It MUST work with Squid proxy. If it does, please add instructions. REOPENing.

PDT agrees P1

Hmm to me it's Worksforme with the Squid proxy on port 80. With Linux nightly 2000090621. I've didn't change anything except of setting the proxy address and port. Maybe it doesn't work on Windows only?

My proxy runs on another (pretty arbitary) port.

And I'm running Linux on the client (see my UA above).

Still worksforme. Here is a list of proxy servers to try - http://proxy.nikto.net/anon_list.htm - I've tried about ten, and they have all worked.

There is some amt of counfusion here. I just used the url john gave above and I cannot do any kind of surfing. I put the above url in the "automatic configuration" textfield. Hit a reload and then I cannot surf. Then I restarted netscape6 and it does not bring up the home page. Is anyone seeing something even similar?

Nitin, that url is a website for you to visit. Then select a proxy server and add it to your http proxy prefs.

John, testing with public proxies is pointless. Setup a private network (with 192.168.0.0 or so addresses) without direct connection to the internet, but only to the proxy. Run standard Squid on port 6543 (i.e. an arbitary one). Set http, ftp and ssl proxy prefs in Mozilla to that machine/port. Open <https://www.fortify.net/sslcheck.html>. Unless this works, don't mark fixed.

Ben: This is really a specific case and a separate bug should be opened and I am closing this since in general proxies work for SSL which is this bug. John would open the other bug but this should be closed.

I disagree that it should be closed. A specific case was given as an aid to the developer to know when they got it fixed. I'm sure its not the only case where there is a problem. I have yet to be able to successfully use mozilla to make a secure connection via the citibank proxy which runs on port 80. (btw, I've been trying mozilla for over a year, and have been anxious to see https over a non-ssl proxy get implemented.) (yes I've got the lastest PSM) This is a dogfood problem for me, since I can't use Mozilla at work because the nature of my work involves secure sites. (I support some credit card web sites and would like to run mozilla to access them to see if we or mozilla have any issues...but I can't - except from home)

Gagan, I'm confused, could you please give a list of "known" working proxies: * Product name, version, platform * which ports they work on (whatever standard ports and two arbitrary) * whether we speak CONNECT or GET Is this working for anyone in a large corporation outside of Netscape?

timeless: I am only going by john's comments above.

Ruslan: can you re-verify this? Seems like people are still having problems with SSL proxy connections. Thx.

This does not work for use within the Sun Microsystems firewall. When we try to go outside through our web proxy servers, SSL doesn't work.

reopening for investigation.

gagan: we have Solaris commercial builds inside the Sun network. If there's anything we can do to help diagnose the problem, please let us know. You're welcome to come to Sun and sit with us on a Windows, Linux, or Solaris box with your favorite build and debugger if you think that would help. If you have any other suggestions about how we can help track down the behavior, please send email to me (drapeau@eng.sun.com) or Rich Burridge (richb@eng.sun.com), the lead engineer for the Solaris port of Netscape 6, or Margaret Chan (mtchan@eng.sun.com), one of the senior engineers on the Solaris port.

I can't make a secure connection through my squid proxy to dick-smiths.

*** Bug 53182 has been marked as a duplicate of this bug. ***

the owner of this bug no longer works for cpd. reassigning to gagan who is the one who assigned to ruslan. Yes, this bug is still a problem. People at sun are still having problems with it and this is a very hot bug for them.

oops. i was misinformed about ruslan. reassigning the bug back to him. Gagan says he should still fix this.

*** Bug 53701 has been marked as a duplicate of this bug. ***

ruslan, could you give some status on this bug please? There have been no comments from you since 31st August. Are you going to be able to fix this in time for nsbeta3? Thanks.

I'm swamped with my own work. I don't think I'll be able to spend sufficient amount of time on it. Looks like though all the proxy settings are propagated correctly (after applying the patch attached) - they get clobbered somewhere and socket transport is still trying to connect to the original host. Back to gagan

Now I'M distributing PSM binaries without a chance to test them, which is ironic. Severity up to blocker.

This doesn't work for me either!

*** Bug 53671 has been marked as a duplicate of this bug. ***

marking nsbeta3- minus as I don't think we should hold PR3 for this. nominating for RTM.

I don't know how to argue more cogently than this, so here goes: PUH-LEEEEEEEZ fix this for PR 3! For any company that sits behind a firewall and uses proxies to get to the outside world (Sun is one example; I'm guessing there are others, but am not personally familiar with any other examples), the existence of this bug prevents those N6/Mozilla customers from using secure sites outside the firewall. It's actually worse than that in the normal case: even for secure sites *inside* a corporate firewall, the browser will often not work. The reason: in enterprises where many prefs are pre-set for the end user, prefs will be set to use a proxy for SSL support, but the corresponding "Ignore proxies for these domains..." pref will not be set. So, even though it's not necessary to go through a proxy to get from one internal site to another, the proxy settings will force the browser to use that proxy, thereby disabling the end user's ability to visit internal secure sites. Imagine trying to do your work at your company without having access to your daily secure web apps (corporate phone book, expense report tool, org chart, secure email, payroll app, benefits chooser, etc.). I can only argue veridically for Sun, but I'm guessing this is a show-stopper for other companies as well. For Sun, it means another two months that the company has to wait before switching to N6 as the default internal browser. With this bug fixed, we have support from the company CEO (more than support, I'd say: significant pressure) to switch to the Netscape 6 browser as the default. To summarize: it'd be Real Nice if this bug were fixed for PR 3. If there's anything I can do to help, I'm willing to try my best.

Sun is definitely not the only company with a proxy requirement. I'd guess (I have no stats at all) that 10-25% of the users are in a similar situation.

Removing nsbeta3- for reconsideration after reading the plea from Sun and many others.

Taking about the list of proxies. Here's what I get: Error 400 Invalid request "CONNECT pc24.patelco.org:443 HTTP/1.0" (unknown method) CERN-HTTPD 3.0A as you see we do issue CONNECT request in that mode (when the pref is enabled - you'll also need a patch attached b4 to this bug report).

*** Bug 52824 has been marked as a duplicate of this bug. ***

A very large percentage of Internet users access the Internet from the workplace for both work related and personal reasons. I can't find a reference but I recall reading a figure around 50%. Most of these users are behind corporate firewalls.

I also belive that PR3 should be able to use PSM behind firewalls!

Ruslan, your changes look ok to me, although I'm unclear on why you need strncasecmps when comparing URI schemes. If you are comparing scheme to scheme, you should use strcasecmp. If you're comparing scheme to full URI string, don't you need to insist on the the : (in case someone used a scheme such as httpsnotreally: at the front of a URI)? /be

Talked to jar and we agree that we agree that the patch is simple enough to consider for PR3 *IFF* we can get some experimental data *TODAY* that the patch fixes the problem and doesn't regress anything else. Who wants to do that? If no one steps up, the fix slips to RTM.

fine. i'll take a look at it and see what I can do.

I can commit engineering resources at Sun to try the patch and see if it works from within our firewall. I'll send private email to PDT with my cell phone number in case you wish to contact me (and apologize in advance for the spam). If I can be helpful here, I'm happy to.

From ruslan's comments at 2000-09-22 17:10, I understand that this patch is *not* supposed to fix this bug. Or wrong?

are we sure that only ssl requests are not going through a proxy. I just setup a proxy server http://oldmonk.red.iplanet.com:8080 and netscape6 could not even go to http://home.netscape.com. I removed the setting from the prefs and I could surf fine.

I have applied the patch on Windows 2000 and rebuilt. Trying to access a secure site the proxy (squid) tells me: 970209736.544 1402 xx.xx.xx.xx TCP_MISS/000 1366 CONNECT www.verisign.com:44 3 - DIRECT/www.verisign.com - So the browser is correctly issuing CONNECT insted of POST. But no further loading of the site itself. In the status line it says Document: Done. I can refresh as often as I would but it will not load. I'm using build 2000092822 with psm fresh installed.

changing product to browser. this isn't a psm/client lib bug. this is a networking bug.. more to follow

so here is what I see: we send: === Start CONNECT aka:443 HTTP/1.1 Host: aka User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.16 i686; en-US; m18) Gecko/20000918 Accept: */* Accept-Language: en Accept-Encoding: gzip,deflate,compress,identity Keep-Alive: 300 Connection: keep-alive === End we get back 200 Connection established from the server... so all is good up to here. then we step up the proxy connection (which goes ok) then we send: === Start GET https://aka/ HTTP/1.1 Host: aka User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.16 i686; en-US; m18) Gecko/20000918 Accept: */* Accept-Language: en Accept-Encoding: gzip,deflate,compress,identity Keep-Alive: 300 Connection: keep-alive === End and from here things go down hill.... we flush the buffer out again incase we have anything, then we get into: nsHTTPRequest::OnStopRequest() [this=426049d8]. wasKeptAlive=0, channelStatus=0 the AsyncRead call in here returns failure, so we call in to ResponseCompleted with a failure status which leads us to report a (bogus) error loading the page. nsHTTPChannel::ResponseCompleted() [this=423fce00] mDataListenet=426b4fb0, Status=20022600017 nsHTTPFinalListener::OnStopRequest [this=426b4fb0], mOnStopFired=0 Error loading URL https://aka/: 804b000f shortly after this point, we get back from the server: === Start HTTP/1.1 404 Not found Server: Netscape-Enterprise/3.6 Date: Fri, 29 Sep 2000 01:01:50 GMT Content-type: text/html Content-length: 207 <...> === End now, if you manage to get the debugger to wait just long enough, you can get the data to come in and so the AsycRead doesn't fail and then we crash in some other pipelining code because it is looking for a request from the request array which happens to be null... this seems to be very timing specific. I've tried to follow what is going on here, but I am having a lot of problems all of the listener, response, etc stuff. I've stepped through all of the PSM proxy code and it is doing what it should be doing. ruslan, I need your help.

Hmm. From what I see there's a bug somewhere when we didn't reset the url to be back a uri, so it should have said GET /, not GET https://aka/ - but it's not a problem after all. The problem is that right after we get Connection established - the transport goes and calls PSM's step up function - that obviously didn't happen as you should not be able to see clear text at all (it should start ssl handshake).

Pavlov, from your analysis, it seems like you are being hit by bug 52492 (which sadly seems to be a rare/difficult to dup problem) I am tempted to ask someone who has this bug as a reproducible problem to try it with pipelining switched off using the pref-- user_pref("network.http.pipelining" , false); Or even the keep-alive with this pref-- user_pref("network.http.keep-alive", false);

added edburns to cc. Gagan, Will this make RTM?

Release noting. "Secure (https) websites such as https://www.verisign.com cannot be reached through a proxy."

I don't suppose we can expect a fix in the trunk in time for an M18 in the next day or two...

Responding to Gagan.. I tried both of those prefs (changed via the gui in debug/networking. Still hung going to a secure site through our proxy server.

I applied the patch and IT WORKS!!!!! WHOOOHOOO!!!! (I'm trying hard not to wake up the whole house - it's 6am here) Thanks!!!! I could sucessfully visit <https://www.fortify.net/sslcheck.html>. It recognies RC4 cipher, 128bit. The test machine can't even resolve inet hostnames, but goes through a Squid 2.3 installation. For the case you missed it: The last patch from Nikolay contains a description of the fix. Nikolay, the standard here are |cvs diff -u|s from the highest common dir. I'll attach such a patch - it contains the same code Nikolay attached.

r=pavlov someone should rtm+ this... gagan?

great! This seems to working well now. If dougt/ruslan can add the sr then I will change this to rtm+ and let PDT approve the final checkin.

Added dougt to cc so he knows he needs to take action to give sr = dougt.

> the final checkin. Note that Nicolay says, POSTs did not work correctly.

This is another bug which should get validated on the trunk before we take the patch on the branch. Also, the patch has a lot of commented out debugging printfs, which should be removed. junruh, please report back in the bug after you've tested this in trunk build.

sr=dougt.

r=ruslan

Just a quick update-- I am in process of getting these changes into the trunk. POST is a known problem and I am yet to find the related bug. Thanks for hanging in there!

Fix has been checked into the trunk. Per PDT's suggestion, I would check this into the branch after ensuring that this works well on the trunk. All of you who have been patiently watching this bug are requested to try the trunk for possible problems and help us test this bug. Thanks!

I blasted through this thread b/c I've to some other fires, but I think I can explain why the "GET HTTPS worked on someone's server". (appoligizes in advance if someone covered this) They are using a Netscape Proxy server that was not configured correctly in a strict sense. There is support for what some mindless docs person named "HTTPS Proxy". It's bascially a feature where the Proxy would support SSL dumb clients by accepting clear GET's to https URL's and then do the SSL as a client when it forward the request. If you want to support this, create a new pref, and call it a "feature", but try explaining the difference between "HTTPS Proxy" and "Security (SSL Tunneling of snews and https) to an end user. It even confused some of our smarter Proxy customers when I supported the product. If someone needs more detail on this, post here or mail me offline.

tever is testing this problem, and he is using build: Win NT 101308 & LINUX (both TRUNK). I've checked the log files of my proxy server (Netscape Proxy Server 3.5x), and it it works. Here are the log entries: 208.12.39.233 - - [13/Oct/2000:12:06:03 -0700] "CONNECT trading.etrade.com:443 HTTP/1.1" 200 0 - 0 0 0 370 73 - - 4 He's still working on the Mac build. When your client sends the "HTTPS Proxy" style request, the logs read: 208.12.39.233 - - [11/Oct/2000:17:57:14 -0700] "GET https://trading.etrade.com/cgi-bin/gx.cgi/AppLogic+Loginpage HTTP/1.1" 403 234 - - - - 304 169 - - -

I tried it early this A.M. and it still hung - doesn't work here. I downloaded PSM on Sep 23 06:11 (am). Do I need a newer one? I had to install it manually using psm_12_rhlinux6.1.tar. My network prefs are: $ grep netw prefs.js user_pref("network.cookie.warnAboutCookies", true); user_pref("network.http.keep-alive", false); user_pref("network.proxy.ftp", "www-proxy"); user_pref("network.proxy.ftp_port", 80); user_pref("network.proxy.http", "localhost"); user_pref("network.proxy.http_port", 8000); user_pref("network.proxy.no_proxies_on", ".oracle.com"); user_pref("network.proxy.ssl", "www-proxy"); user_pref("network.proxy.ssl_port", 80); user_pref("network.proxy.type", 1); localhost:8000 is junkbuster which forwards to www-proxy. The same exact settings work with ns4.

Try a newer version of PSM from http://docs.iplanet.com/docs/manuals/psm/psm-mozilla/ Scroll down to the bottom and click on the correct button.

This worksforme on the trunk builds. I was able to connect to various https sites through an ssl proxy with no problems. Ben verified that the CONNECT is showing up in his log files. Checked 1013 and 1016 builds on winNT, Linux, and Mac. I am also seeing the problem with posting but I don't know what the bug # is.

> Ben verified that the CONNECT is showing up in his log files. Not me, so I guess, he means Benjamin Chuang.

yeah, I meant Ben Chuang ... sorry.

Based on tever's (and benc's) feedback I am setting this back to rtm+ for PDT approval to allow me to check in these changes for the branch.

opened bug 56935 for the POST problem. If there is a dupe of this feel free to mark it as such.

I realise that bug 56935 has been opened to handled the POST case, but I'll make this comment here as most people interested in this are already on the cc: for this bug. The same person who has fixed 31174 (Nikolay Igotti) is now working on a fix for the POST case. He's got it working with: https://webbranch.technologycu.com/ and his own test server, but he needs more URL's for servers to test against (particularly ones where he will be able to get access to the server logs as well. If you can help here, please add your comments to this bug or email me. Thanks.

If I go to https://login.yahoo.com/, I get an error (alert box) right away that says "The connection was refused when attempting to contact login.yahoo.com". And in the xterm window where I started mozilla, it says: Error loading URL https://login.yahoo.com/config/login?.src=ym&.last=&promo=&.intl=us&.bypass=&.partner=&.done=: 804b000d we don't handle eBorderStyle_close yet... please fix me And there is no psm process running. ssl proxy and http proxy are both set (and those settings work in ns4). keepalive and http pipelining are both disabled. I am running 2000101221 build. I cannot install a newer PSM due to the install bugs, and since psm process is not even started, I have no reason to believe it would help.

Qa > tever

Is it true that automatic proxy configuration does not work and the manual configuration works fine in the edit|preferences? I could not do anything when I setup auto-configuration, but manual config works fine for me.

nitinp, do you have the right bug? Your comments would seem to be relevant to bug 53080

This bug is decision starving :-(

Junruh emailed me a zip file with psm. I unzipped it and struggled with the start-psm script and finally got it working. Now I can get to https://login.yahoo.com/ but I still cannot log into yahoo mail. I assume this is the POST problem. Can you folkes at netscape/mozilla get that POST fix into the nightly build so we can try it?

PDT says rtm++, okay to land non-POST fixes on branch. Please nominate POST bug separately, as appropriate.

I'd like to make a general note here: For the purposes of verifying that your proxy client implementation is correct, I think that nobody should be testing with a PAC file. Too many ways it can add complexity. I see from the previous thread that this has happened.

is there anything keeping this from landing?

it has landed :) (on the branch as well now) Since the post is being tracked in a separate bug I am going to go ahead and close this one.

verified on branch using server frame.packetgram.com:443 WinNT 101908 Linux rh6 101909 Mac os9 101908 already tested on trunk

*** Bug 50141 has been marked as a duplicate of this bug. ***

Bad news, SSL via proxy is not working correctly in build 2000112704 (Win98). Our firewall is a Raptor 1.0. When I try to connect to https://www.verisign.com, I get prompted for my firewall password and, after entering it, the web page starts to load. However, it looks like the HTTP headers are being included with the HTML. After most of the page loads, the browser freezes and I have to kill it.

Similarly, mozilla freezes when trying to go to https://login.yahoo.com/ As in your case, it starts to load the page, and then it dies. (btw: I have not been able to get talkback to work through our proxy - it does not seem to support authentication. thus, i've not been able to send any of the talkback crash files)

forgot to mention, we are using an authenticated netscape proxy server on port 80 for https to direct traffic through our corporate firewall.

Tom Canova: try turning off TLS support and see if https://login.yahoo.com/ works. I just tried it on Win32 and it worked for me if I turned off TLS, crashed N6 if I left TLS turned on. To turn off TLS support, go to the Tasks menu and choose "Tasks->Privacy and Security->Security Manager" to bring up the Personal Security Manager panel. Once that appears, choose the "Advanced" tab, then choose the "Options" tab. You'll see three choices for SSL settings: SSL 2, SSL 3, and TLS. Un-check TLS, close that window, quit the browser, then re-start it. Try again, you should be fine. I don't know why this is the case, but it is.

If turning off TLS supports fixes the problem, then that means the server implements TLS incorrectly. The web site administrator should be notified. TLS is the lates IETF standard for SSL (TLS == SSL v3.1). We should avoid encouraging users to turn it off because most web sites will not fix their problem unless users make them aware of it.

We shouldn't crash, no mather what the server does. Can somebody please file a bug? As long as we do crash on random SSL sites, it is very reasonable to turn it off. Is that the only problem? Can we close the bug again?

If there is crashing using TLS with "SSL Proxy" but not by itself, then there is a problem on ourside. If there is a problem with using TLS without "SSL Proxy", then this should not have been mentioned here :)

Currently I have PSM version 1.3. Under the advanced tab, it only lists SSL2 and SSL3 (not TLS) I used the Debug->Install PSM menu to get to the psm-mozilla page on docs.iplanet.com. I think I have the current. https seems to work inside our firewall. The 2 or 3 sites I've tried outside our firewall have all crashed the browser.

My browser crashes when going to http://login.yahoo.com/ or https://www.amazon.com/exec/obidos/checkout-sign-in/... Amazon and Yahoo seem like pretty mainstream sites...

Tom Canova: you're out of luck for the moment. PSM 1.3 doesn't allow you to turn off TLS support; there's no UI for it, nor a preference that we were able to find. If you're using the Solaris port of Netscape 6 PR 3, you'll get a new version that will let you turn off the TLS option in about a week (~December 15), when we switch to PSM 1.4. The released versions of Netscape 6 already let you do this preference change. If you're using Mozilla, then I'd recommend asking how to compile PSM 1.4 and using it with your Mozilla build; it will at least let you work around the problem for now.

> If you're using Mozilla, then I'd recommend asking how to compile PSM 1.4 Or get mine <http://www.bucksch.org/1/projects/mozilla/dist/psm>.

This bug was marked to be fixed in a previous milestone but it didn't get fixed properly. Nominated for beta1.

Marking fixed again per previous comments. Problems mentioned for reopening are irrelevant to this bug.

agree - marking verified

Man, I hate to do this, but in the last two days' commercial builds of Netscape 6, I can no longer go to www.amazon.com and access my account through the secure server. In fact, no HTTPS functionality seems to work. I realize that this is Netscape 6 versus Mozilla, but these are nightly builds based on the Mozilla code base. Has anybody verified in the last few days that HTTPS via web proxies still works? I claim not, but would be jubilant to be proven wrong. (build id on Win2K: 2001020704)

I can confirm this going through a squid 2.3 proxy with build: 2001020704 user-agent: Mozilla/5.0 (Windows; U; Win98; en-US; m18) Gecko/20010207

ugh! any chance a fix will materialize in time for 0.8? This is a high profile regression. updating status whiteboard and keywords and stuff.

I don't mean to panic anybody. It's probably just me. It should be fairly easy to verify, though; I'll do what I can at Sun (get others to use the same build on a different Windows box, and also Linux), and perhaps somebody at Netscape can verify as well.

I'm using Linux and Squid, and https doesn't work for me either. Requests seem to time out. drapeau, you have secure servers within your firewall, IIRC. Please try to access them.

See bug 66904. I've reported it there because I didn't want to reopen this bug.

From the dup bug: ------- Additional Comments From deh@meganet.pt 2001-02-06 16:41 ------- Copying from my report of bug #67815 (which was a dupe of this one, sorry): Mozilla 2001020508, Linux (Mandrake 7.2, glibc 2.2.1, kernel 2.4.0) using squid 2.4 (recent cvs snapshot) The proxy works perfectly for Netscape 4.7x, MSIE and Mozilla 0.7 clients. Using the aforementioned daily build (2001020508), when trying to access any secure page (e.g. https://www.fortify.net/sslcheck.html), it takes a long time, about 1 or 2 minutes, and then says "The operation timed out attempting to contact https://(address)". If I configure Mozilla to not use a proxy server for HTTPS sites, it works. The problem is only when using the proxy. ------- Additional Comments From Tom Mraz 2001-02-06 23:33 ------- I see exactly the same behavior except the proxy is squid 2.3.STABLE4

*** Bug 66904 has been marked as a duplicate of this bug. ***

Okay, I tried accessing a Sun-internal HTTPS site two ways: 1) Do not use any proxy servers. Result: works fine. 2) Using proxy servers for both HTTP and SSL. Result: no worky.

I too see George's problem. I built the trunk on WINNT4.0 SP4 from source on last Friday with BUILD_PSM=1 and I can't access my etrade account.

If you configure a proxy for HTTP and no proxy for HTTPS, it also works... the problem is just when using HTTPS through a proxy.

The last nightly that worked for me was the 23rd (Build 2001012304).

Worked for me on win32 commercial as late as 2001-02-05-11, but definitely broken on win32 comercial now (2001-02-08-08-Mtrunk). Proxy server logs show: 208.12.40.178 - - [08/Feb/2001:14:00:59 -0800] "CONNECT www.verisign.com:443 HTT P/1.1" 200 0 - 0 0 0 275 73 - - 31 The request is in quotes, and looks good. However the next 4 fields basically show that after making a successful client-proxy connection, the client never completes a successful request-response transaction with the destination server. This could be due to an SSL handshake failure or other problems. I'll keep digging, but getting info out of SSL is hard to do sometimes :) Also, since this was working in several releases, would it be too much to ask that we put the regression in a separate bug?

*** Bug 68100 has been marked as a duplicate of this bug. ***

A tcpdump trace seems to show to me that after getting the CONNECT response the client simply does nothing. Perhaps it doesn't parse the (correct) response right, or is internally deadlocked. 2001020810/Linux.

Is there movement on this bug? It's a 0.8 bug so I'm trying to make sure that there's still traction.

darin: can you investigate if the asyncwrite changes you landed have something to do with this regression? I can't think up of anything else that may have changed from networking side. If you don't find anything I'd let someone from security group (javi?) investigate this.

My AsyncWrite changes went in on 1/23 and broke PSM, but this was fixed on 1/27. So if this was working on 2/5, then this most likely isn't a regression caused by my AsyncWrite changes. bryner made a change on 2/6 that might be related: bug 67582 - set channel on security info object for ssl-forcehandshake sockets.

who should own this?

I doubt it's my checkin, this just sets an additional attribute on the security info object. Can anyone confirm whether nsSocketTransport::doConnection is calling ProxyStepUp() on the socket?

darin, gagan, this seems to have stalled. can one of youhelp get it moving again.

Can someone point me to an SSL proxy for testing?

benc has an ssl proxy server running for testing purposes frame.packetgram.com 443

Reliably, you can only test on a system which is not connected to the Inet other than through the proxy. Netscape *should* have such systems.

Ok... We are successfully making the CONNECT with the SSL proxy. It returns a 200 success code. At this point, we switch into SSL mode (via ProxyStepUp). After doing so, we enter PR_Poll waiting for socket i/o. For some reason, PR_Poll never wakes up the socket. I switched back to necko (1/22/2001) and found that everything worked just fine, which means that this is definitely a bug in necko and not in PSM. I'm trying to compare the behavior of the two necko's in order to hopefully locate the bug.

I have a fix for this.

Beware, the patch I just submitted fixes this bug but breaks FTP (see bug 65220). The correct fix requires a true bidi socket transport, which dougt has on his branch.

re: ben's comments above I'm not clear on the exact configuration you are referring to. Do you mean we should have a system that is on a proxy-firewalled subnet? I'm not sure how that helps in this situation. If you want to make sure that you client is not connecting directly, there are methods of verifying the correct behavior without the mentioned configuration...

> If you want to make sure that you client is not connecting directly Yes. > there are methods of verifying the correct behavior without the mentioned > configuration... OK. (Read the backlog of this very bug for why I am so distrusting.)

r=dougt on the latest patch

Please note: my patch is a workaround fix. This bug should not be considered resolved once this patch lands. For some reason, polling on a SSL socket is broken. PR_Poll sometimes doesn't wakeup. This is especially true after ProxyStepUp is called. PR_Poll never wakes up if it is entered right after calling ProxyStepUp. My patch ensures that PR_Write/PR_Read get called first before entering PR_Poll, and this seems to do the trick (at least for https://www.fortify.net/sslcheck.html). However, my patch only seems to fix the problem for some HTTPS sites. In particular, I am still unable to login to sourceforge.net and hotmail.com using an SSL proxy. I can do this in moz 0.7, but not with my trunk build. I even tried running a trunk build with necko from moz 0.7, and that didn't work. Which, confirms for me that this is not a necko bug. In these examples, the last thing necko does is successfully write some data to the socket and then enter PR_Poll. PR_Poll times out after a while and we get PR_POLL_HUP on the socket. Any ideas? Who should own this?

cc'ing mstoltz and ddrinan for SSL help

- if (mSuspendCount) { + if (mSuspendCount) mSuspendCount -= 1; [auto-pilot style nitpicker engaged] what's wrong with the -- unary operator? + //} else { + // // Only a suspended transport can be resumed... + // rv = NS_ERROR_FAILURE; + //} + Rather than [just] commenting this code out, how about an XXX comment raising some issues? This seems like a big red flag to me, just based on a cursory reading, and I'd like to know more. I'm trying to find SSL experts by phone. Please let me know if you get ahold of one. Do you need protocol-level-snooping analysis, or is it possible to debug this (into the PSM code as necessary)? Yes, we're holding mozilla0.8 for this. /be

*** Bug 65968 has been marked as a duplicate of this bug. ***

Oops... my patch is a bit sloppy. I will clean that up. I am still waiting to hear from someone with SSL knowledge. I am going to try to generate a PSM log at least.

*** Bug 67465 has been marked as a duplicate of this bug. ***

adding nelsonb to cc list. He knows a thing or two about SSL. ;)

nothing more happens until PR_Poll times out.

this log is for necko as of mozilla 0.7 attaching a second one with the latest necko including my patch

jgmyers says this might be caused by bug 66706. trying work around suggested there.

The work-around may have to be applied in PSM since necko wouldn't be calling the SSL'ified socket. If that's the case, insert the work around at the following line of PSM: mozilla/security/psm/server/sslconn.c line 940

The workaround jgmyers (call PR_Write(fd, 0, 0) right after ProxyStepUp) suggested still didn't get me anywhere. Any other suggestions?

Adding to CC

I nominate this bug report for the following awards: 1. Most reopened. (6 times) 2. highest number of distinctly separate problems reported in a single bug report (only superficially similar). 3. highest number of "additional comments". 4. Most attachments 5. Most speculation about how it's supposed to work 6. Most speculation about why it's not working. 7. Most speculation on how to fix it. Both of the two Necko/PSM logs show that an SSL connection was completed, and the application data written on the socket, so bug 66706 is irrelevant. A proxy would not be expected to log the (possibly encrypted) traffic on the socket after the CONNECT command succeeded, though it might log the amount of such data. What's really needed here is the output of a packet trace or possibly the output of the ssltap program, although that might cough on the CONNECT command.

that's odd, my sr comments from this morning didn't take. I must not have been logged into bugzilla and didn't pay attention after the submit. Sorry about that. the socket work around looks okay to me (just looked at the revised one with Brendan's comments). Although I'm confused by Javi's comments. He says the patch needs to be moved into psm. does that mean we aren't working around it in all cases or something? I guess I'm a bit confused as to were we stand. Are we happy that this work around to the socket transport makes things happy (at least for mozilla .8)? With a real fix coming later? Or do we need to change some bits in psm?

Javi's point was that the "woraround" suggested by John Myers applies to SSL sockets (that is, NSPR sockets that have had SSL attached to them, such as is done inside PSM 1.x, and is not done in the browser process). So, the workaround would have no effect in the browser, as Darian confirmed. That "workaround" is also not needed in PSM. John's "workaround" is intended to get the handshake started in cases where it has not started. But both PSM trace logs clearly showed that the SSL handshake completed.

amazing that we lived this long without adding the flags for readwrite case. Great find darin! r=gagan but I'd still like to make sure that there were no reasons for not doing this earliar. rpotts can you sr this?

r=bryner on final patch

Comment on the last patch. Aren't we setting the mSocketFlag in the eSocketState_Connected state in Process()? Why do we have to re-OR the flags back in? Are we totally skipping this state?

Wow -- pretty fundamental. So I guess the async i/o control flow never goes through the Connected state in nsSocketTransport::Process? Is that right? Looks like mscott sr'd most of it. I just dug through lxr to understand the async read/write control flow better, and also to grok how PUT/POST data goes through a second AsyncWrite. So I can stamp a rubbery sr=brendan@mozilla.org, and plead with anyone who is around and can help to get this into MOZILLA_0_8_BRANCH. Many thanks to darin, and to everyone who helped to figure this out. Please watch the milestone schedule, don't give us a last minute thrill ride next time, ok? We'll try to keep mozilla #qa on top of proxied SSL regressions, too. The slashdotters will love us again. /be

As the original filer of this bug, you all can imagine how I feel about it these days. ;-) Are all of you folks done with it now?

i can't get that patch to apply cleanly :!! D:\2001\08\mozilla\netwerk\base\src>patch < D:\2001\08\31174.patch patching file `nsSocketTransport.cpp' Hunk #10 FAILED at 2208. 1 out of 10 hunks FAILED -- saving rejects to nsSocketTransport.cpp.rej

Patch checked into .8 branch.

re tquan@spamcop.net: this bug is going to die and stay dead real soon. If you must get your V-day gift early, asa already gave me permission to close this bug when it's fixed and force each new problem or regression into a new bug. Bug 53080 has more attachments, and it will meet a similar fate once it works in a milestone.

*** Bug 68694 has been marked as a duplicate of this bug. ***

mozilla 2001021503 linux: Going to https://digitalid.verisign.com does work, but consumes lots of cpu power and considerable amounts of memory. It becomes slower as I reload it. Going to https://sourceforge.net takes so much time and memory I had to kill it (over 222Mb). Seems like a leak and a delay while a new connection is estableshed. It takes some time, then the page is there, and then it takes a lot of time and the images show up.

As I told Asa the patch doesn't completely fix SSL proxy. To do so, we need to wait for dougt's branch to land. My patch just makes SSL proxy work well for simple sites (such as hotmail.com) and very poorly for complex sites (such as sourceforge.net). We end up busy waiting a lot with this patch because the socket transport does not _really_ support bidirectional i/o. This is exactly the same problem FTP currently has.

busy-waiting doesn't account for the memory growth -- what's the cause of that, and is there a band-aid for 0.8? /be

*** Bug 69029 has been marked as a duplicate of this bug. ***

0.8 (Linux) becomes a lot slower and grows A LOT memory-wise when doing HTTPS-through-proxy. Wasn't it supposed to wait for a true fix? :(

add myself to cc list

add myself to cc list

Well, I have squid as a transparent proxy, and SSL over it with 0.8 seems to work quite fine, with me. I do not have conditions to reproduce the situation in a more closed situation... but since my isp is using transparent proxy, I don't have problems. This again adds to the idea that mozilla is speaking wrongly to the wrong ports.

I'm getting this with squid-2.3, Build-ID 2001021912: 82620410.587 29951 170.56.121.156 TCP_MISS/000 39 CONNECT www.verisign.com:443 - DIRECT/www.verisign.com - and it just sits there. here's the tcpdump: [root@uw7moz /root]# tcpdump -i eth0 -v host 170.56.121.156 Kernel filter, protocol ALL, datagram packet socket tcpdump: listening on eth0 23:14:57.553107 < 170.56.121.156.1407 > uw7moz.squid: S 649072503:649072503(0) w in 8760 <mss 1380,nop,nop,sackOK> (DF) (ttl 126, id 15887) 23:14:57.553280 > uw7moz.squid > 170.56.121.156.1407: S 3419165269:3419165269(0) ack 649072504 win 31740 <mss 1380,nop,nop,sackOK> (DF) (ttl 64, id 37786) 23:14:57.604790 < 170.56.121.156.1407 > uw7moz.squid: . 1:1(0) ack 1 win 9660 (D F) (ttl 126, id 15893) 23:14:57.636429 < 170.56.121.156.1407 > uw7moz.squid: P 1:275(274) ack 1 win 966 0 (DF) (ttl 126, id 15894) 23:14:57.636560 > uw7moz.squid > 170.56.121.156.1407: . 1:1(0) ack 275 win 31740 (DF) (ttl 64, id 37791) 23:14:57.826025 > uw7moz.squid > 170.56.121.156.1407: P 1:40(39) ack 275 win 317 40 (DF) (ttl 64, id 37795) 23:14:57.983404 < 170.56.121.156.1407 > uw7moz.squid: . 275:275(0) ack 40 win 96 21 (DF) (ttl 126, id 15911) at this point it stopps.

My patch was not applied to the trunk (it was a hack for mozilla 0.8) This bug is correctly fixed on dougt's DOUGT_CHANNEL_CHANGES_02112001_BRANCH which will be merging on the trunk very soon.

Updating target milestone to 0.9

I applied the patch(attchment 25317). Before applying the patch, I can't even get into the secure site. After applied the patch, the browser can get into the secure website, however, it just hangs during loading the page. This is what I observed with the latest code from trunk.

Dougt's branch landed. Marking FIXED.

*** Bug 69894 has been marked as a duplicate of this bug. ***

Verified with the 2/23 Win32 trunk, Mac mozilla, and Linux trunk builds.

*** Bug 69481 has been marked as a duplicate of this bug. ***

For the record, and as a warning: In 0.8 Linux, SSL over proxy caused mozilla-bin to grab memory without bound at a rapid rate. I had to kill it at 223MB, or it would have crashed my machine.

I have also seen exactly this behaviour in Moz 0.8 on NT4. (Grabbing memory without bound at a rapid rate after going to SSL site though a proxy, had to kill from Taskmanager.) Is there already a new bug on this?

This is fixed in current daily builds. However, most people will keep using 0.8 until there's a new "official" release, so maybe a 0.8.1 or something, with this fixed, would be a good idea...