User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:2.0b7) Gecko/20100101 Firefox/4.0b7 Build Identifier: Mozilla/5.0 (Windows NT 6.1; rv:2.0b7) Gecko/20100101 Firefox/4.0b7 Askmen pages from 30th Jan build onwards are not giving up memory contributing to a steep memory hike, see also bug https://bugzilla.mozilla.org/show_bug.cgi?id=631494 Reproducible: Always Steps to Reproduce: 1.note down memory usage of FF 2.access http://www.askmen.com/specials/2010_great_male_survey/ after loading for 15 seconds, note down memory usage 3.click Dating & Sex tab, note down memory after 10 seconds, 30 seconds 1 minute. Before 30th build memory drops around 90-100MB after 30 seconds, from 30th build onwards the memory does not drop, but slowly increases. Actual Results: (tab below refers to Dating & Sex tab) FF 4 11Pre 26 01 start 68MB after 1 minute, increase to 89MB with askmen page, click tab, increase to 189MB, after 30 seconds fall to 97.9MB FF 4 11Pre 28 01 start 69.7MB after 1 minute, increase to 94.6MB with askmen page, click tab, increase to 190MB, after 32 seconds fall to 99.9MB FF 4 11Pre 29 01 start 71MB after 1 minute, increase to 93.2MB with askmen page, click tab, increase to 190.2MB, after 32 seconds fall to 98MB FF 4 11Pre 30 01 start 69MB after 1 minute, increase to 101MB with askmen page, click tab, increase to 212.8MB, after 70 seconds rise to 213.7MB, after 60 rise to 214.5MB, 30 to 215.6MB FF 4 12Pre 04 02 start 70MB after 1 minute, increase to 100.5MB with askmen page, click tab, increase to 191.1MB, after 50 seconds rise to 196MB, further small increases Expected Results: Memory should be freed up after clicking tab

I should add, the increase of 90MB-100MB for the tab seems excessive, between FF4 Beta 6 and Beta 7 the memory usage for clicking the tab doubled from around 44MB to 90-100MB and continued in that vein b8,9,10 and 11Pre. FF 3.6.13 has much hardly any memory hike in loading the page, has a quick 50MB hike with the tab, but gives up the memory much much quicker 5 seconds compared to 35-80 seconds FF 3.6.13 start 47.3MB after 1 minute, 45.5MB with askmen page, with tab increase to 97.1MB, then fall back to 51.1MB after 5 seconds FF 4 Beta 1 start 47.3MB after 1 minute, increase to 83.3MB with askmen page, after 15 seconds fallback to 55.9MB, with tab increase to 108.1MB, then fall back to 66.7MB after 65 seconds FF 4 Beta 2 start 58MB after 1 minute, increase to 70MB with askmen page, with tab increase to 119MB, then fall back to 79.8MB after 35 seconds FF 4 Beta 3 start 50MB after 1 minute, increase to 60MB with askmen page, with tab increase to 110MB, then fall back to 72MB after 30 seconds FF 4 Beta 4 start 56.4MB after 1 minute, increase to 73MB with askmen page, with tab increase to 118MB, fall back after 80 seconds to 82.2MB FF 4 Beta 5 start 61.1MB after 1 minute, increase to 78.8MB with askmen page, with tab increase to 124.1MB, fall back after 45 seconds to 86.7MB FF 4 Beta 6 start 56MB after 1 minute, increase to 74.2MB with askmen page, with tab increase to 118MB, fall back after 80 seconds to 82MB FF4 Beta 7Pre 20100914031635 start 70.4MB after 1 minute, increase to 88.6MB with askmen page, after 30 secs increase to 103.8MB, with tab increase to 140.3MB, fall back after 33 seconds to 128.6MB, after 15 seconds to 108.2MB FF4 Beta 7Pre 20100916030904 start 70.7MB after 1 minute, increase to 87.5MB with askmen page, with tab increase to 126.4MB, fall back after 44 seconds to 111.3MB, after 10 seconds to 94MB FF4 Beta 7Pre 20101001071914 start 72.6MB after 1 minute, increase to 92.4MB with askmen page, with tab increase to 151.5MB, fall back after 64 seconds to 98.3MB FF 4 Beta 7 start 73MB after 1 minute, increase to 100MB with askmen page, with tab increase to 191MB, fall back after 85 seconds to 107.7MB FF 4 Beta 8 start 74.4MB after 1 minute, increase to 99.8MB with askmen page, with tab increase to 192MB, fall back after 35 seconds to 114MB FF 4 Beta 9 start 65.5MB after 1 minute, increase to 96MB with askmen page, with tab increase to 193MB, fall back after 35 seconds to 102MB FF 4 Beta 10 start 65MB after 1 minute, increase to 90.2MB with askmen page, with tab increase to 185.5MB, fall back after 35 seconds to 96.2MB FF 4 11Pre 26 01 start 68MB after 1 minute, increase to 89MB with askmen page, with tab increase to 189MB, after 30 seconds fall to 97.9MB FF 4 11Pre 28 01 start 69.7MB after 1 minute, increase to 94.6MB with askmen page, with tab increase to 190MB, after 32 seconds fall to 99.9MB FF 4 11Pre 29 01 start 71MB after 1 minute, increase to 93.2MB with askmen page, with tab increase to 190.2MB, after 32 seconds fall to 98MB FF 4 11Pre 30 01 start 69MB after 1 minute, increase to 101MB with askmen page, with tab increase to 212.8MB, after 70 seconds rise to 213.7MB, after 60 rise to 214.5MB, 30 to 215.6MB FF 4 12Pre 04 02 start 70MB after 1 minute, increase to 100.5MB with askmen page, with tab increase to 191.1MB, after 50 seconds rise to 196MB, no further change

Was beta 7 when jaegermonkey landed? It uses a lot more memory, which may explain some of the increase. See this bug: https://bugzilla.mozilla.org/show_bug.cgi?id=615199

Ah thanks, that explains that issue. Any idea why the memory is released so much quicker & more efficiently in 3.6.13 after loading?

See my comment on bug 631494 https://bugzilla.mozilla.org/show_bug.cgi?id=631494#c21 with latest build, and dating from 30th build onwards, there is steadily increasing memory with Askmen Top 99 Women page http://www.askmen.com/specials/top_99_women/ eventually reaching 660MB private bytes usage with 20 or so clicks through the pictures, ABP disabled, and 712MB with ABP enabled, and memory is never released

Found two other News Corp sites affected by the 29th-30th change, Sky TV shop, and News of the World, where the memory continues to increase with browsing & is never released, see tests on bug 631494 https://bugzilla.mozilla.org/show_bug.cgi?id=631494#c24 http://www.sky.com/shop/tv/ http://www.newsoftheworld.co.uk/notw/public/home/

This is a general problem not confined to News Corp sites, also found to be affecting: all http://abclocal.go.com sites ABC NEWS slideshows i.e. http://abcnews.go.com/International/slideshow/egyptian-protesters-clash-police-12785329 National Gallery paintings http://nationalgallery.org.uk/paintings/collection-overview/ see https://bugzilla.mozilla.org/show_bug.cgi?id=631494#c25 for tests, these bugs probably need to be merged and marked and as hardblocker

After Image loading, memory usage is high: See http://forums.mozillazine.org/viewtopic.php?p=10396313#p10396313 Low memory usage: http://hg.mozilla.org/mozilla-central/rev/ba84db82eed5 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b11pre) Gecko/20110128 Firefox/4.0b11pre ID:20110129124939 High memory usage 2 times before: http://hg.mozilla.org/mozilla-central/rev/3ea2b5a7c9c8 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b11pre) Gecko/20110129 Firefox/4.0b11pre ID:20110129131351 Pushlog: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=ba84db82eed5&tochange=3ea2b5a7c9c8

(In reply to comment #7) > After Image loading, memory usage is high: > See http://forums.mozillazine.org/viewtopic.php?p=10396313#p10396313 > > Low memory usage: > http://hg.mozilla.org/mozilla-central/rev/ba84db82eed5 > Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b11pre) Gecko/20110128 > Firefox/4.0b11pre ID:20110129124939 > High memory usage 2 times before: > http://hg.mozilla.org/mozilla-central/rev/3ea2b5a7c9c8 > Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b11pre) Gecko/20110129 > Firefox/4.0b11pre ID:20110129131351 > Pushlog: > http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=ba84db82eed5&tochange=3ea2b5a7c9c8 Not quite sure what you are saying but I found 28th build to be near identical usage to 29th, stays between 210-250MB during abc local slideshow showing the fire, it is with the 30th builds onwards that the memory steadily increases to 400MB+ (all addons disabled manually, hw accel on)

Bug 631951 will greatly reduce JaegerMonkey's memory usage, if it makes it into Firefox 4.0 (I sure hope it does). That doesn't explain the difference between the Jan 29 and Jan 30 builds, though.

The range in comment 7 actually makes sense. smaug, do we have existing bugs on that? If there was a change from the 29th to the 30th, I'd like to see a regression range from that.

I suspected that Bug 624549, Don't call GC so aggressively in nsJSContext::CC, r=gal, a=jst Jan 29 13:10:51 2011 was the most likely cause, would that not have been applied in the 30th build, and not the 29th? These are the two builds I observed the steep change in: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2011-01-29-03-mozilla-central/ http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2011-01-30-03-mozilla-central/ Full list of affected sites discovered so far: http://www.askmen.com/specials/top_99_women/ http://www.askmen.com/specials/2010_great_male_survey/ http://www.sky.com/shop/tv/ http://www.newsoftheworld.co.uk/notw/public/home/ all http://abclocal.go.com sites http://abcnews.go.com/International/slideshow/egyptian-protesters-clash-police-12785329 ABC NEWS slideshows http://nationalgallery.org.uk/paintings/collection-overview/ National Gallery painting collection

Just curious.. But is AdBlock Plus installed on any of the profiles you tested with? If so, https://bugzilla.mozilla.org/show_bug.cgi?id=631494 might be related.

I do have it installed, but I have done tests with it completely disabled & enabled, posted on the other bug you linked to: https://bugzilla.mozilla.org/show_bug.cgi?id=631494#c21 https://bugzilla.mozilla.org/show_bug.cgi?id=631494#c24 there is no real difference with ABP enabled/disabled to the observed change bar a slight additional memory footprint with ABP (20-30MB or so) between the 29th and 30th builds onwards for the symptom of a lack of regular memory release causing a steep increase in memory usage on the above linked sites.

Can we get some QA love here on reproducing the problem?

I am able to reproduce this on a new profile with no extensions installed. Disabling all JavaScript in the preferences will stop the leak from occurring, as the memory usage remain normal. If this script is being allowed to run the leak will occur: http://www.askmen.com/includes/js/am/global.js.php Sticking the JS-file in a basic HTML-page will increase the memory usage if you reload the page a few times. <html> <head> <title></title> <script type="text/javascript" src="global.js.js"></script> </head> <body> </body> </html> The script is quite big though, 170 kB covering 930 lines of minimized code.

As of this evening I have testing infrastructure available that can reliably test for bugs in which we don't GC via the nsJSEnvironment when we should. I will see if I can distill this down into a testcase.

Jonath, QA has reproduced this issues. However we have been tracking it in bug 631494. I really think this is a dup of that bug. Can someone give reasons why it is not?

I was asked to create this bug separately as I observed a distinct memory increase on various sites without ABP enabled after the 29th builds (i.e. from 30th onwards). Most of the work on the other page seem to be with ABP enabled, I am not entirely sure everyone is aware on that bug of the non ABP problem with all the other sites I have listed in comment 11.

Duping per Matt's observations above.

I have reopened this as the patches for bug 631494, although they 80% fix the ever increasing memory issue, have not fixed the issue of memory never being released if stopping browsing or switching to another tab: https://bugzilla.mozilla.org/show_bug.cgi?id=631494#c87

CC/GC scheduling is now different than it used to be. Does the memory usage stay higher than before when browsing Askmen and then waiting for a minute or so?

Yes, I did tests on askmen, Sky TV shop, News of the World, ABC Local and National Gallery sites and they will only drop 5-10MB at the most with latest builds (from an already higher level than 29th & previous builds), whereas with 29th and previous builds it would drop 80-100MB after 15, 35 or 80 seconds- I waited over 160 seconds and there was no significant release of memory for any of these sites (with all addons disabled).

So on trunk memory usage goes up from initial 75MB to 250MB when browsing through 20 askmen pages, and after some time it drops to 170MB. That sounds like a similar drop you saw before 29th. Are you testing the very latest builds? Something which has 88799325812a?

I last tested the 15th build, will download todays and post back.

I tried build from 29th and memory usage ended up to 270MB, not to 170MB what I get today's build.

OK here are my tests with Windows 7 32 bit, all addons disabled, shows 29th build releases much more memory in comparison to latest nightly build: FF 4 11Pre 2901 2 tabs + askmen gms tab, 58.6MB after 1 minute, increase to 89.9MB with askmen great male survey page, click tab, increase to 182.5MB, after 32 seconds fall to 90.3MB FF 4 12Pre 1702 2 tabs + askmen gms tab, 62.6MB after 1 minute, increase to 87.3MB with askmen great male survey page, click tab, increase to 184.8MB, no change after 140 seconds, if switch tab drop by 9MB to 175.9MB, no change after 140 seconds FF4 Beta 11pre 2901 2 tabs + askmen 99 women tab, after 1 minute 55.4MB, open askmen top 99 women, peak 104.5MB, fall to 89.1MB, click start with 99, peak 123.6MB, fall to 105.6MB, click next rise to 207.5MB, then 220.4MB, then 227.5MB, then 217.1MB, then 216.8MB, then 221MB, then 226.1MB, then 230MB, then 232MB, then 229MB, then 233.4MB, then 228.6MB, then 234MB, then 236MB, then 237MB, then 233MB, then 230MB, then 235MB, then 234MB, 16MB drop after 10 seconds to 217MB, after 25 seconds fall to 135MB FF4 Beta 12pre 1702 2 tabs + askmen 99 women tab, after 1 minute 57.3MB, open askmen top 99 women, peak 92.5MB, fall to 83.5MB, click start with 99, peak 108.4MB, fall to 105MB, click next rise to 192.3MB, then 195.6MB, then 194.8MB, then 195.8MB, then 194MB, then 193.9MB, then 194.9MB, then 195.4MB, then 201.1MB, then 196.9MB, then 198.7MB, then 198.3MB, then 197.4MB, then 197.3MB, then 196.4MB, then 195.1MB, then 198.6MB, then 198.6MB, then 195.4MB, no change after 140 seconds, if switch to another tab no change FF4 Beta 11pre 2901 2 tabs + Sky TV shop http://www.sky.com/shop/tv/ after 1 minute 55.8MB, open Sky TV shop, rise to 87.2MB, click Entertainment tab, jumps to 183.4MB cycle through sub tabs, and other TV tabs & sub tabs, i.e. Sky Sports, Movies, HD TV, Anytime, Sky 3D, Extra Channels, Included Channels, stays around 186-230MB, after stop browsing fall to 101.9MB after 32 seconds FF4 Beta 12pre 1702 2 tabs + Sky TV shop http://www.sky.com/shop/tv/ after 1 minute 57.4MB, open Sky TV shop, rise to 84.8MB, click Entertainment tab, jumps to 173.1MB cycle through sub tabs, and other TV tabs & sub tabs, i.e. Sky Sports, Movies, HD TV, Anytime, Sky 3D, Extra Channels, Included Channels, stays around 176-220MB, after stop browsing no fall below 182MB after 140 seconds, if switch to another tab only drop 5MB

And just to clarify, which beta12pre build are you using? What does about:buildconfig 's "Built from" say? Is it something *after* http://hg.mozilla.org/mozilla-central/rev/88799325812a

I just did an auto update a couple of hours ago, this is the link given in about:buildconfig http://hg.mozilla.org/mozilla-central/rev/94a51a3b64d4 Version: 14.00.50727.762

You pasted the link from Jan 29 build ;)

(In reply to comment #29) > You pasted the link from Jan 29 build ;) Doh, yeah I tested the 29th last is the reason :D, just updated again & I assume the version on the auto updater is still the same http://hg.mozilla.org/mozilla-central/rev/4b9e814fe3ab

Smaug, we need to figure out whether this still happens, or if the recent GC scheduling changes took care of this...

I can't reproduce this using the step from https://bugzilla.mozilla.org/show_bug.cgi?id=631494 (without any addons), but I need to test other sites still. orangezilla, could you perhaps give exact steps which links to try and on which sites. Note, as I'm not from US I have no idea what for example site "ABC Local" actually is.

(In reply to comment #32) > orangezilla, could you perhaps give exact steps which links to try and on > which sites. Note, as I'm not from US I have no idea what for example site > "ABC Local" actually is. Hi, all sites are listed in comment 11, I have this bug page open and http://input.mozilla.com/en-US/beta/search?q=&product=firefox&version=4.0b11&date_start=&date_end=&sentiment=sad (not eternally pessimistic, honest ;)) as well as the site in question (3 tabs in total) for the Askmen great male survey(gms) I follow steps in https://bugzilla.mozilla.org/show_bug.cgi?id=631733#c0 Sky TV shop I browse the tabs and sub tabs from left to right as in the description in comment 26 Basically you just browse the sites as described, then stop browsing, in 29th build and previous 80-100MB would be released between 15-80 seconds after stopping browsing, 30th build up till now, no significant memory is released if you stay on the tab for 140 seconds+, if you switch tab (i.e. to this bug) it releases 5-10MB if anything. The only exception to that, for whatever reason is the ABC NEWS slideshow which releases about 80MB if you switch tab.

Un-surprisingly bug 593426 seems to have affected the behavior. If I change image.mem.min_discard_timeout_ms back to 10000 and open a new tab, memory usage will drop to the level it used to be. (This is while testing http://www.askmen.com/specials/2010_great_male_survey/)

orangezilla, could you perhaps try to set image.mem.min_discard_timeout_ms to 10000.

So, I think, the original memory usage growth was because of bug 624549, but that was then fixed in bug 630932. But before the latter was fixed, the patch for Bug 593426 was pushed to m-c and that causes us to keep image decoded longer. But still investigating some more, since I'm seeing memory usage to drop pretty soon even without backing out the patch for Bug 593426 and opening a new tab, yet keeping the askmen tab open in the background.

No, Bug 593426 may not still be the problem. Apparently when the "Dating & Sex tab" is clicked, more memory is allocated, but that doesn't get freed until *GC* is run after the last CC (the one which collected 0). Quite strange case. I don't see memory leaks, but memory is just not released as soon as earlier. Investigating if we could do tiny tweaking to GC/CC scheduling.

(In reply to comment #34) > Un-surprisingly bug 593426 seems to have affected the behavior. > If I change image.mem.min_discard_timeout_ms back to 10000 and open a new > tab, memory usage will drop to the level it used to be. > (This is while testing http://www.askmen.com/specials/2010_great_male_survey/) Thanks for continuing to investigate this, I tried the setting you suggested (image.mem.min_discard_timeout_ms to 10000) but I can't didn't see any change with the askmen site on first 17-02 nightly build.

This brings back the user activity observer.

I uploaded that patch to tryserver.

Should it be here, can only see up to 16th builds for you so far? http://ftp.mozilla.org//pub/mozilla.org/firefox/tryserver-builds/

The builds will be here http://ftp.mozilla.org/pub/mozilla.org/firefox/tryserver-builds/opettay@mozilla.com-19cef2916e56/ Atm only linux builds, but others should be coming there soon.

Comment on attachment 513439 [details] [diff] [review] possible patch Andreas, what do you think about taking back part of the user activity observing. This way we would try to release harder when user is inactive. 5 is perhaps too much, 3 might be enough.

Some tweaking still. Don't keep up calling GC/CC often when user is inactive (if something is collected), but just call it 3 times. That should be enough to release memory (so that taskmanagers show less memory usage) if pages are quite static, and if pages are not static, GC or CC will be called anyway. I know, this is hackish.

(In reply to comment #42) > The builds will be here > http://ftp.mozilla.org/pub/mozilla.org/firefox/tryserver-builds/opettay@mozilla.com-19cef2916e56/ > > Atm only linux builds, but others should be coming there soon. OK I tried the Win32installer.exe http://ftp.mozilla.org/pub/mozilla.org/firefox/tryserver-builds/opettay@mozilla.com-19cef2916e56/tryserver-win32/firefox-4.0b12pre.en-US.win32.installer.exe different behaviour definitely observed with this build, after clicking the askmen tab, the memory falls to original level after 20 seconds, does the same for all tabs on that page.

Andreas, just increasing the timeout doesn't seem to help. Investigating some more.

Andreas, actually, several GCs may be needed. Not all native objects are CCollectable, yet GC can hold to such objects. So a gc may release native object which is itself holding some js objects...or am I wrong here? The '3' is just random which happens to work reasonable well, and shouldn't cause too many pauses to animations running while user is inactive.

I have tested the other sites in comment 11, all drop 80-100MB after 20 seconds of ceasing browsing with Olli's new build with the exception of National Gallery site, only if you stop on a picture page with a picture zoomed in and fully loaded, where it does not seem to release memory if you stop panning/zooming, however, if you switch tab it will drop 80MB after 100 seconds or so.

That National Gallery behavior could have something to do with image.mem.min_discard_timeout_ms

(In reply to comment #49) > That National Gallery behavior could have something to do with > image.mem.min_discard_timeout_ms I currently have it set to 120000, should try 10000 again?

Note: discarding has no effect on sites you're currently looking at. It affects only background tabs.

Andreas, there is also the case that if gctimer runs while there is some js on stack (that could happen with sync XHR, alert(), etc). In that case GC wouldn't be able to collect all the possible objects.

Not trying to give you a hard time I just really want to understand the cause. Should we poke the GC when a nested event loop unwinds?

Andreas, if you have any hints how to debug this... I'll probably try valgrind to see what is actually released.

I think I have a fix for this, see bug 617569. Taking this bug.

Comment on attachment 513832 [details] [diff] [review] patch r=me for landing sooner, Smaug feedback still needed. /be

Memory still doesn't seem to be freed when opening and closing Google Documents

Benjamin, please file a new bug for that issue with exact steps to reproduce. CC me and Andreas. Thanks!

Filed bug 636220

(In reply to comment #64) > Memory still doesn't seem to be freed when opening and closing Google Documents A few things should remain cached.

No no no... this bug is NOT fixed. Using the browser is a pain now.

dE: file a new bug with STR. Writing "No no no..." here is not helpful. This bug had a patch that addressed its symptoms at least in part. One patch landing per bug is best for auditing when backing out, so new bug is due even if this bug's general symptom was not fully fixed. /be

(In reply to comment #69) > dE: file a new bug with STR. Writing "No no no..." here is not helpful. This > bug had a patch that addressed its symptoms at least in part. One patch landing > per bug is best for auditing when backing out, so new bug is due even if this > bug's general symptom was not fully fixed. > > /be Already done that.