User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.0.1) Gecko/20060214 Camino/1.0 Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.0.1) Gecko/20060214 Camino/1.0 This is somehow a reopen of bug 153884 in a specific page. When I try to see the source of a web page generated dynamically by a WebObjects application server, the request is re-submitted to the application server, with the same consequences as described in bug 153884. I don't known if this is specific to WebObjects applications, but I cannot manage to reproduce this otherwise, for example by crafting a PHP page copying the same content, headers, port, etc.. as those sent by the WebObjects application. If that helps, I have seen this happen both when the application runs as a stand-alone WebObjects application server, or when it runs inside Tomcat as a servlet. Reproducible: Always Steps to Reproduce: 1. Go to a website dynamically generated by a WebObjects application server, such as www.frontbase.com. 2. Start any tcp packet listener, such as tcpdump, to monitor HTTP traffic. (I used sudo tcpdump -i en0 -s0 -X 'tcp port 80' ) 3. On the previously loaded page, right-click and choose "View Frame Source" or "View Page Source", depending on what is available. Actual Results: Notice that the packet listener captures a new request sent to the application server, mirroring the one used to get the original page. (You can repeat steps 3-4 several times if you wish) Expected Results: I expect the source of the original page to be displayed without triggering a duplicate request, which can have undesirable side-effects (see bug 153884). My user.js contains the following prefs, but it doesn't seem to be related: user_pref("nglayout.initialpaint.delay", 0); user_pref("network.http.pipelining", true); user_pref("network.http.proxy.pipelining", true); user_pref("network.http.pipelining.maxrequests", 100);

The frames at that site are dynamically generated. The big tip-off is the fact that they're running WebObjects. One of the frames in the referenced URL: mark@crack bash$ curl -I 'http://www.frontbase.com/cgi-bin/WebObjects/FrontBase.woa/wa/homePage?currentPage=HomePage&currentMenu=homepageMenu&currentModule=HomePage&action=homePage' HTTP/1.1 200 Apple Date: Wed, 29 Mar 2006 15:27:42 GMT Server: Apache/1.3.33 (Darwin) cache-control: private cache-control: no-cache cache-control: no-store cache-control: must-revalidate cache-control: max-age=0 expires: Mon, 06-Mar-2006 03:05:30 GMT pragma: no-cache content-length: 9086 Content-Type: text/html All of that cache-control and expires and pragma stuff means "don't put this in your cache!" Unfortunately, Camino's view source implementation hits the cache, and it can't use what's not there. ("HTTP/1.1 200 Apple"? OK, guys...)

(In reply to comment #1) > All of that cache-control and expires and pragma stuff means "don't put this in > your cache!" Unfortunately, Camino's view source implementation hits the > cache, and it can't use what's not there. > Maybe I'm missing something, but while testing this I created a PHP page that copies the headers sent by my WebObjects application, and when I access that one with Camino, there is *no* duplicate request. Here is the output of curl -I on my PHP page: $ curl -I http://localhost/test.php HTTP/1.1 200 OK Date: Wed, 29 Mar 2006 15:41:26 GMT Server: Apache/1.3.33 (Darwin) PHP/4.4.1 X-Powered-By: PHP/4.4.1 cache-control: private cache-control: no-cache cache-control: no-store cache-control: must-revalidate cache-control: max-age=0 expires: Wed, 29-Mar-2006 12:57:18 GMT pragma: no-cache x-webobjects-loadaverage: 1 set-cookie: wosid=z021lvkpSspQ9Neykn58I0; version="1"; path=/ set-cookie: woinst=-1; version="1"; path=/ Content-Type: text/html; charset=UTF-8 Thus it does *not* seem that the cache-control/expires/pragma stuff is enough to trigger this problem! However if I manually empty the Cache (option-command-E), I can indeed reproduce the issue on any page. Another thing: I have tried this on Firefox 1.5 (20060303), and I cannot reproduce this issue, either on frontbase.com or with my PHP page. But I *do* see the duplicate request on Firefox if I manually empty the cache! I guess the re-submit is acceptable when the user manually empties the cache, but I do not think it is otherwise!

So what's the deal with this? Is Fx misbehaving, and this is INVALID, or...?

I'm having a problem that might be similar with PHP. When a page with a form is loaded, a PHP script adds an entry to a MySQL database with the next ID number (it increments up). When a form is submitted, that entry is supposed to be updated with new content. I have the ID displayed on the page before the page is submitted. In Firefox and Safari, when I view-source, it'll show me the same ID in the source. However, when I view-source in Camino, the next ID number is displayed and I see that a second entry has been added to the database, as if the page was loaded again. I think this is a problem for web developers where content is dynamically generated and is context-sensitive. You want the source to load exactly what you're looking at, so it should get the source from a local copy of the page, not download a new copy. I could see this being a problem where people are displaying dates. You have the current date on the page and if you view source it later, it'll be a totally different date yet if you go back to the page it'll still have the old date.

Bart, what headers are your pages returning? curl -I 'http://url.to.the/page.php'

I am having a similar problem to this with a custom PHP CMS I am developing. However, there are no frames involved. The problem I have is I login to the CMS, then if I view source of a page that I am logged into, the source shows me the login screen again. It's as if the separate request to retrieve the source is not sending any cookies or other information? This might be a completely different bug, but it makes me have to use Firefox and/or Safari whenever I need to actually see the source of the CURRENT page, *AS IT IS*, not as a new request. I don't quite understand why the view source goes through the cache? Does FF do this? Why doesn't it just view the source of the current document or frame? It'd be a bonus if it actually showed the current HTML as per the DOM instead of the source "downloaded". Anyway, if this is a completely unrelated bug, I'll open up a new one. To try this bug out, go to any site that requires you to login. Login, then view source. For me, it shows me the source of the login page (or a variant of an error page saying you need to login). In any case, you are no longer logged in. To make this even more confusing, if you go to a site that has a "remember me" type functionality, it seems to send those cookies, but not cookies that expire at the end of the user's session.

Can a test case use server-side scripting? Because in this case it is somewhat required. In any case, the simplest test I can think of is use your favorite server-side scripting language to output the time. When viewing source, the time should always show what is seen in the browser, and not the current time retrieved from a new server request. PHP: <?php date('r'); ?> If this is not adequate, please tell me what I can do to make a proper test case?

(In reply to comment #7) > Can a test case use server-side scripting? Because in this case it is somewhat > required. In any case, the simplest test I can think of is use your favorite > server-side scripting language to output the time. I made a page like this at: http://www.workofbart.com/caminotest.php The left frame is printing out a time with PHP. If you view a source on that frame, the date should be the same. I wonder if this bug has just been fixed or is in the nightlies (I switched to the stable version after having some problems) because I can't reproduce the bug anymore even on a website I previously had problems with.

> frame, the date should be the same. I wonder if this bug has just been fixed or > is in the nightlies (I switched to the stable version after having some > problems) because I can't reproduce the bug anymore even on a website I > previously had problems with. Sorry, it turns out I actually am using a nightly from June 3rd so maybe this has been fixed?

I think this issue is critical for web developers, and as such should be considered blocking for camino 1.1. Could someone please mark this bug as CONFIRMED, since it is very easy to verify? And since a valid testcase is needed, here is one using a WebObjects application my company developed : 1.- Go to this URL: http://www.athenaweb.org 2.- Notice that even though the login box hides it partially, a random video snapshot is displayed in the center of the home page, in the 'VIDEOS' section, with a description underneath. This video changes every time you refresh the page, you can try it. 3.- Pay attention to the title and description of the video currently displayed. For example the title could be 'Electricity at home'. 4.- Now right-click on the page and choose 'View frame source' 5.- Search in the HTML source code for the title of the video, you will not be able to find it (unless it is contained in other parts of the page, but this is not likely). This is not the source code of the actual page you are looking at, but the source code of a new request sent to the server! 6.- To confirm this, scroll down in the HTML source and look for the part of the page that contains the video's description. You can search for 'Library_HomeVideoPicture', which is currently the CSS class of the DIV element containing the picture. A few lines below this DIV, you will find the text description of the video, along with the 'Length', 'Producer' and 'Country' (if available for that video). 7.- Verify that the description is not the same as the one on the actual page. 8.- Now you can try to repeat steps 1 to 7 using Firefox, and you will see that the title and description of the video is actually identical in the actual page and in the source code. The source code also loads much faster, because no request needs to be sent to the server. For reference, the headers sent by the AthenaWeb home page are the following: $ curl -I http://www.athenaweb.org/ATWP/WebObjects/ATWP.woa/wa/ATWPDA/home HTTP/1.1 200 OK Date: Thu, 22 Jun 2006 09:48:08 GMT Server: Apache/1.3.33 (Darwin) mod_jk/1.2.4 mod_ssl/2.8.22 OpenSSL/0.9.7g PHP/4.3.11 Cache-Control: max-age=60 Expires: Thu, 22 Jun 2006 09:49:08 GMT Set-Cookie: JSESSIONID=4D49E616D3CD16CC8B03FD5FFD71C57D.athenaweb-2; Path=/ATWP cache-control: private cache-control: no-cache cache-control: no-store cache-control: must-revalidate cache-control: max-age=0 expires: Thu, 22-Jun-2006 02:11:25 GMT pragma: no-cache x-webobjects-loadaverage: 4 set-cookie: wosid=n5k985gt4IrYMVkvlZC1Uw; version="1"; path=/ATWP/WebObjects/ATWP.woa set-cookie: woinst=-1; version="1"; path=/ATWP/WebObjects/ATWP.woa x-webobjects-servlet: YES Content-Length: 29133 Connection: close Content-Type: text/html;charset=UTF-8

The test cases here (comments 8 and 10) both WFM with the 2006052022 branch nightly. Olivier, are you able to reproduce this with a branch nightly, and if so can you also reproduce it with a clean profile?

(In reply to comment #11) > The test cases here (comments 8 and 10) both WFM with the 2006052022 branch > nightly. Olivier, are you able to reproduce this with a branch nightly, and if > so can you also reproduce it with a clean profile? As you requested, I have tried to reproduce this with a nightly build, on my desktop (iMac CoreDuo 20" 2GHz). In fact I tried it with each of the following versions, just in case: - 2006021400 (1.0) - 2006061318 (1.0.2) - 2006042704 (1.0.1) - 2006062601 (1.0+) -> latest nightly It was not working. Then I renamed my profile files (~/Library/Application Support/Camino and ~/Library/Preferences/org.mozilla.camino.plist, is there anything else?) to test with a clean profile. I tested again with each version, and no, it still wasn't working! (the only difference with the nightly is that the source opens in a window instead of a tab) BUT, and this is where it gets weird, I have tried it also on a Powerbook G4, which has Version 2006042704 (1.0.1)... and there it *works*! The difference is actually very easy to spot: when the 'frame source' tab or window opens, if the status bar briefly shows 'Read www.athenaweb.org', then it's working and reading from cache, but if it shows 'Waiting for www.athenaweb.org', and then 'Transferring data from...', then it's requesting the page again! I couldn't believe it, so I transferred the Camino.app sitting on the PB to my desktop. I ran the test again, and surprise... it was *not* working! So I'm lost here... is it something that can only be reproduced on Intel Macs, or am I missing something obvious, maybe when cleaning my profile? Any thoughts?

(Following up to my own comment #12) After noticing the behavior differences between Camino on Intel and PPC, I did one more test: I forced Camino on my desktop computer (Intel iMac) to run through Rosetta, and did the same tests. Guess what, the problem could not be reproduced anymore, and even though Camino wa much slower, it displayed the correct source for web pages (and the status bar was saying 'Read ...' instead of 'Transferring data from ...')! So I can confirm that this bug is reproducible on any Camino version (tested using 1.0, 1.0.1, 1.0.2, and the 2006062601 nightly), but ONLY ON INTEL MAC! This should explain why people had trouble testing/reproducing this issue. (Not sure if the 'Hardware' field should be changed to 'PC', as there is no x86 Mac option) I hope that we can now proceed on actually confirming this bug, and quickly fixing it.

Wevah was able to repro the athenaweb case (comment 10) on his MBP, so confirming this. Mento, any idea why this is different on Intel? Some lurking cache bug?

(In reply to comment #13) > This should explain why people had trouble testing/reproducing this issue. Except that I haven't been able to reproduce it on my Intel iMac, so it's not quite as simple as that.

Olivier, anything "unique" about your setup that might be influencing this? When you were testing this, did you always have a clean profile, or did you set some prefs first? I forgot Stuart was on Intel, so there's clearly an additional element here that we need to identify....

(In reply to comment #16) > Olivier, anything "unique" about your setup that might be influencing this? > When you were testing this, did you always have a clean profile, or did you set > some prefs first? What I did to test is described in my comment #12 : I tried with my original profile (whose non-default prefs are listed in the bug description), and also with what I thought was a clean profile (what I did is simply rename the profile files I could find, ~/Library/Application Support/Camino and ~/Library/Preferences/org.mozilla.camino.plist, and it seemed to clean the profile). In both cases I could reproduce it, but not anymore as soon as I forced it through Rosetta. Apart from that, well.. I can't really think of anything unique to my setup that could impact Camino. How about the MBP on which you were able to reproduce this? I suppose it would be something on the software side... I'm running 10.4.6, but was probably using another minor version in march. Maybe if I had an idea about what to look for? What kind of 'uniqueness' could impact that piece of code, except profile preferences and HW platform?

We need more info here; -> 1.2 for now

Now I'm seeing this on at least some pages; I'll try to investigate more.

I'm seeing this also. Using 1.1a2, viewing source on a page where I've logged in shows the source for the login page. This happens in a LOT of places-- basecamp is a good example, but it's really anywhere that needs a login. Viewing source should view the source for what's already been rendered, not resubmit, as other people have said. I definitely believe this should be considered a blocking bug. This is a HUGE problem.

(In reply to comment #20) > I definitely believe this should be considered a blocking bug. It isn't. There is no way we would block 1.1 for a bug that isn't new, is only intermittently reproducible, and won't affect the majority of users. The most useful thing someone could do to help get this fixed is try to isolate the conditions that trigger it, as I have a lot of trouble getting it to happen on demand (although I certainly have seen it occasionally).

This isn't an intermittent problem for me-- it happens 100% of the time on sites with logins when I'm running Camino 1.1a2 natively on Intel. Force it into Rosetta, it doesn't happen. I haven't yet found a site on which I need to login where viewing the source reflects the current state of the page. vbulletin sites, gmail, this bugzilla, basecamp, my website's CMS (textpattern), etc... In Rosetta, the problem just doesn't happen. Intel native, it always happens.

Looking at the source it appears we basically just drop frame source requests on the floor in terms of trying to load them from cache, so the fact that some frame sources seem to be using the cache is probably an accident. I have no idea why the symptoms would go away on PPC. We need a better way of getting page descriptors that lets us get sub-frame descriptors as well.

Minusing since this seems like it's going to be a very complicated fix this late in the game. Stuart, if I've misunderstood your previous comment, and if you're planning to work on this soon, feel free to renominate.

I maintain that this bug is very critical. It makes viewing source almost useless on many, many sites. I'm surprised it wasn't in 1.5, but is a 1.5 maintenance release (like 1.5.1 or 2) really unrealistic?

(In reply to comment #25) > I maintain that this bug is very critical. It makes viewing source almost > useless on many, many sites. Please stop trying to argue to have the severity changed on this bug. The current classification is correct, and it's not going to change. > I'm surprised it wasn't in 1.5, but is a 1.5 > maintenance release (like 1.5.1 or 2) really unrealistic? If someone fixes it, then no, it's not unrealistic. No-one has made any statement that it's unrealistic, so I'm not sure what you are arguing against. Please read the bugzilla etiquette guide before commenting here again. Complaining in bugzilla, and thus causing me to get pointless email, will not result in me having more time to look into this.

Mass un-setting milestone per 1.6 roadmap. Filter on RemoveRedonkulousBuglist to remove bugspam. Developers: if you have a patch in hand for one of these bugs, you may pull the bug back to 1.6 *at that point*.

Gets the descriptor for frame requests as well as top-level requests. A few notes about testing this: - The bug isn't always reproducible before the patch (although it's not clear to me why). - The source windows always shows activity in the status bar suggesting it's doing a reload even when it clearly isn't (i.e., when the bug doesn't happen). - This doesn't address the no-cache case (bug 392017) so some cases still won't work. Which all together means this is a pain to test. However, I've verified the actual code paths taken, and we are now following the page descriptor path in every case, so we are now doing the right thing for frames at this layer which we definitely weren't before.

Comment on attachment 276737 [details] [diff] [review] fix r=me. As Stuart mentioned, the cache-controlled test cases (frontbase.com, athenaweb.org, gmail, ...) will not be fixed by this patch and instead depend on bug 392017.

Comment on attachment 276737 [details] [diff] [review] fix sr=pink

Landed on trunk and MOZILLA_1_8_BRANCH.

Not sure how well this will back-port, but nominating for 1.5.2 to see.

Landed on CAMINO_1_5_BRANCH.