User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 [Note: This is reproducable in both Mozilla 1.7.5 and Firefox 1.0.] On pages that return a Last-Modified header but no Expires header, Mozilla will automatically generate its own Expires date. Problem is, if you're viewing a page that hasn't been modified in months/years, the auto-generated date is set far into the future. Here's an example URL: http://cr.yp.to/daemontools/readproctitle.html I visited this page today (January 10, 2005). If I look at the Page Info I see: Modified: Monday, July 16, 2001 12:33:03 AM Expires: Wednesday, May 18, 2005 2:44:58 AM Note how the Expires date it has generated is over 4 months (!) into the future. This means that if, for example, this page happens to be updated next month, and I come back, I won't see the changes because the cache isn't going to check for changes until May. Isn't that excessive? Shouldn't there be an upper bounds on how far into the future expiration dates are set (e.g. 1 day)? Reproducible: Always Steps to Reproduce: 1. Visit http://cr.yp.to/daemontools/readproctitle.html 2. Go to Page Info, and look at the "Expires" date. Actual Results: Wednesday, May 18, 2005 2:44:58 AM Expected Results: Something sooner, e.g. January 11, 2005

if the page hasn't been modified in 3.5 years, I don't think it's unreasonable to assume it won't change in the next 4 months... and expiring it on the next day already seems _way_ too early to me.

We follow RFC 2616's recommendation. From section 13.2.4: Also, if the response does have a Last-Modified time, the heuristic expiration value SHOULD be no more than some fraction of the interval since that time. A typical setting of this fraction might be 10%.

That said, a hard upper limit on freshness lifetime is not unreasonable. We could even make it pref controlled for extra fun. BTW, you can force the browser to validate each cached document at least once per session by setting the following pref: browser.cache.check_doc_frequency = 0 That may help if you visit a lot of sites like this.

> if the page hasn't been modified in 3.5 years, I don't think it's unreasonable > to assume it won't change in the next 4 months... Okay, granted, it's somewhat of an extreme example. However, this problem isn't limited to pages that haven't been updated in years; I've found it to be a problem for pages updated as often as once a month. I've gotten many complaints from Firefox users about it. As I understand it, to get the freshness lifetime, the current code subtracts the Last-Modified date from the current date, then divides the result by 10. Thus, if one visits a page that was last modified one month ago, the Expires date will be set 3 days in the future. But what if the page happens to be updated the next day? Firefox users will have to wait upwards of _3 days_ to see the update, unless they're savvy enough to hit Refresh in their browser (and in many cases they'd have to be psychic to even know there's an update waiting!). Internet Explorer users, however, will see the update just fine (as IE by default checks for updates once per session).

IE checks automatically. It's once per session option is not the default. RFC 2616 gives origin servers plenty of ways to control the freshness lifetime for a page. They can set "Cache-control: max-age=86400" to force the user agent to revalidate the page once per day. Perhaps Apache and other popular servers should default to setting a "reasonable" max-age value for all content instead of relying on user agents to guess what they mean.

> IE checks automatically. BTW, I really don't know what they mean in their preferences by "automatically", but I presume it means follow the RFC, given that once-per-session is a separate option.

(In reply to comment #3) > BTW, you can force the > browser to validate each cached document at least once per session by setting > the following pref: > browser.cache.check_doc_frequency = 0 > That may help if you visit a lot of sites like this. I actually tried that setting, but it did not have any effect. It still would not issue any new requests to the web server after closing & re-opening the browser. (I assume that's what "once per session" is supposed to mean?) #221036 ("Cache "once per session" not honoured") seems to cover this specific problem.

(In reply to comment #6) > > IE checks automatically. > > BTW, I really don't know what they mean in their preferences by "automatically", > but I presume it means follow the RFC, given that once-per-session is a separate > option. Actually, both "Automatically" and "Every time you start Internet Explorer" check for updates to pages at the start of each new session. (Pages that previously returned a Last-Modified header and no Expires header, that is.) According to http://www.jsiinc.com/SUBF/TIP2800/rh2895.htm , "Automatically" is an extension of "Every time you start Internet Explorer" that affects whether the browser checks for new _images_ at the start of each session. If Firefox(/Mozilla) could check for updates to pages at the start of every session as IE does, that would really be ideal...

This is an automated message, with ID "auto-resolve01". This bug has had no comments for a long time. Statistically, we have found that bug reports that have not been confirmed by a second user after three months are highly unlikely to be the source of a fix to the code. While your input is very important to us, our resources are limited and so we are asking for your help in focussing our efforts. If you can still reproduce this problem in the latest version of the product (see below for how to obtain a copy) or, for feature requests, if it's not present in the latest version and you still believe we should implement it, please visit the URL of this bug (given at the top of this mail) and add a comment to that effect, giving more reproduction information if you have it. If it is not a problem any longer, you need take no action. If this bug is not changed in any way in the next two weeks, it will be automatically resolved. Thank you for your help in this matter. The latest beta releases can be obtained from: Firefox: http://www.mozilla.org/projects/firefox/ Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html Seamonkey: http://www.mozilla.org/projects/seamonkey/

(In reply to comment #9) > This is an automated message, with ID "auto-resolve01". Same behavior exists in Firefox 1.5 Beta 1.

In case it's at all helpful, here's a patch I came up with a while back that imposes a (hard-coded) upper limit on how far into the future auto-generated Expires dates can be.

-> default owner

Confirming, based on many bugs listed in Bug 328605.

FWIW, Squid has a similar mechanism for capping heuristic freshness.

the default squid rule seems to place a max of 3 days.. seems as good as anything for our heuristic cap.

well, lets round to a week.

Comment on attachment 8712354 [details] [diff] [review] autogenerated expires needs max Review of attachment 8712354 [details] [diff] [review]: ----------------------------------------------------------------- r+ with comments (or updates) ::: netwerk/protocol/http/nsHttpResponseHead.cpp @@ +414,5 @@ > // freshnessLifetime = max_age_value > // <or> > // freshnessLifetime = expires_value - date_value > // <or> > +// freshnessLifetime = min(1week,(date_value - last_modified_value) * 0.10) one_week :)) because it looks like "LWEEK" @@ +462,5 @@ > LOG(("last-modified = %u, date = %u\n", date2, date)); > if (date2 <= date) { > // this only makes sense if last-modified is actually in the past > *result = (date - date2) / 10; > + const uint32_t kOneWeek = 60 * 60 * 24 * 3; isn't it 3 days..?

> > + const uint32_t kOneWeek = 60 * 60 * 24 * 3; > > isn't it 3 days..? you can tell I waffled on that :)

https://hg.mozilla.org/integration/mozilla-inbound/rev/5c871eddec4991abdd901a18ea821a1fe2878b9d bug 277813 - autogenerated expires needs max r=mayhemer