Benjamin Xiao Reporter
	Description • 14 years ago

User-Agent:       Mozilla/5.0 (Windows NT 6.1; rv:2.0b13pre) Gecko/20110223 Firefox/4.0b13pre
Build Identifier: Mozilla/5.0 (Windows NT 6.1; rv:2.0b13pre) Gecko/20110223 Firefox/4.0b13pre

Opening up a Google document and then closing will cause an increase of memory around 10-15 MB.

I have the Google Docs file manager, Gmail, and Facebook as app tabs. This takes about 130-150 MB of memory at startup. I open up a Google document, wait a while, close it, and memory rises to 167 MB and doesn't go down for several minutes. Do this again and again, and memory rises to 180-200 MB.

Reproducible: Always

Steps to Reproduce:
1. Open a Google doc in a tab
2. Close the tab
3. Notice a jump in memory usage of about 10-15 MB
4. Repeat steps to see a bigger memory increase.
Actual Results:  
Memory is not freed after closing a Google document

Expected Results:  
Memory is freed after closing a Google document

	Brian Carpenter [:geeknik]
	Comment 1 • 14 years ago

I'm seeing this with the above STR on with the latest nightly on Windows 7 Pro x64.

	Andreas Gal :gal Assignee
	Comment 2 • 14 years ago

If you do this repeatedly, does memory constantly grow? or does it top out?

	Andreas Gal :gal Assignee
	Comment 3 • 14 years ago

Until we can show that this is a leak (constant memory growth), this won't block FF4 and I won't have cycles to work on it after until FF4 ships, so if you guys can provide some more data it would help assess the severity and escalate this bug if needed.

	Alex R.
	Comment 4 • 14 years ago

I just tested this on 4b11. Here's what I see:

- Upon opening a document in Google Docs, js/mjit-code increases 10-20MB in about:memory.
- Closing that document and opening a new one quickly results in another 10-20MB increase, and this can be done repeatedly to get 60MB+ of memory use.
- Closing the tabs and waiting for a few minutes results in the js/mjit-code use dropping significantly, close to the original levels.

The only part of this that surprises me is that the mjit-code kept around from one Google Doc can't be reused for the next. It looks like the memory is freed eventually, though.

(I don't know if this is valid with the latest nightlies, though)

	Benjamin Xiao Reporter
	Comment 5 • 14 years ago

(In reply to comment #2)
> If you do this repeatedly, does memory constantly grow? or does it top out?
With the app tabs I've mentioned previously, closing all Google Docs causes the memory to stay around 240-250 MB constantly. It does not grow out of control, but 250 MB is quite a bit of usage for just 3 app tabs, don't you think? I understand that things might be cached, but won't this be detrimental to systems with smaller memory?

I am doing more tests to see if my app tabs are related. The following results are from a fresh browser session with no app tabs.

Startup with Reddit.com as homepage: 136 MB
Load up Google Doc file manager and one document: 160 MB
Close file manager and document, and reload reddit.com: 140 MB
Reopen file manager and document: 162 MB
Close file manager and document, and reload reddit.com: 142 MB
Reopen file manager and document: 162 MB
Close file manager and document, and reload reddit.com: 141 MB

Since the results showed that Google Docs is looking relatively okay, I decided to test each of my app tabs individually. The Google Doc file manager, Gmail, and Facebook all released memory properly. Gmail did however raise the memory usage to 150 MB when it was unloaded and reddit.com was loaded back up. Subsequent reloading and unloading of Gmail did not increase the memory usage past 150.

I think there's something going on when all 3 app tabs are loaded at the same time. Is this just a caching issue and nothing to worry about?

	Benjamin Xiao Reporter
	Comment 6 • 14 years ago

I have a strong feeling this bug could really be bug 635121, in which case this bug is a dupe. I am now testing with just Google Docs file manager and Gmail as app tabs. Will report back on how that goes.

	Benjamin Xiao Reporter
	Comment 7 • 14 years ago

So I've done a few more tests with just the file manager and Gmail as app tabs.

Initial startup with two app tabs: 150 MB
Load up two google docs (one Word doc and one PowerPoint): 222 MB
Close both docs: 207 MB
Reopen both docs: 230 MB
Close both docs: 210 MB
Reopen both docs: 235 MB
Close both docs: 210 MB

So it seems like it increases memory usage a bit at the beginning and then after a while it stops growing.

Not sure if this is counts as a leak or its just a memory cache thing.

	Benjamin Xiao Reporter
	Comment 8 • 14 years ago

Hmm okay, I just experienced a session where I had 3 Google Docs open and memory usage went up to 350MB. Closing the documents sent it down to 270 MB, but that's 70 MB more than expected.

I was also using collaborative editing, which may or may not have had an effect. I'll do more testing to see if this matters.

	Ed Morley [:emorley]
	Comment 9 • 14 years ago

Were you running NoScript and ABP at the time?
If so, see also bug 639186.

	Benjamin Xiao Reporter
	Comment 10 • 14 years ago

No, I have no extensions and a fresh profile. The memory usage does not grow out of control, but it takes up more than it should when tabs are closed.

	Nicholas Nethercote [inactive]
	Comment 11 • 13 years ago

Benjamin, Nightly builds now allow you to manually trigger garbage collection from about:memory.  Can you try a Nightly build (from http://nightly.mozilla.org), and (a) see if the problem is still there -- a number of memory usage improvement have happened since 4.0 betas -- and (b) if so, try triggering a manual GC (with the "GC+CC" button at the bottom of about:memory).

It would be good to have the "explicit" measurements from about:memory when you do this.  Thanks very much!

	Nicholas Nethercote [inactive]
	Comment 12 • 13 years ago

Per-compartment reporters (bug 661474) will likely help diagnose this better.

	Nicholas Nethercote [inactive]
	Comment 13 • 13 years ago

Oh, bug 666058 has a good chance of fixing this one.  It sounds like fragmentation.

	Justin Lebar (not reading bugmail)
	Comment 14 • 13 years ago

I just tested this with a new build.

I opened a fresh browser, then loaded 4 google docs pages in new tabs, then closed the tabs.  I measured memory usage immediately after closing the tabs and also a few minutes after.  I reduced memory usage before the last two measurements.

Here are the cherrypicked about:memory results.  Numbers below are [startup / after closing tabs / after waiting a few minutes], in mb.

 * explicit                 [44 / 122 / 100]
 * resident                 [70 / 172 / 147]
 * resident - explicit      [26 /  50 /  47]

The ~25mb increase in [resident minus explicit] is strange.  Hopefully bug 674290 will give us some insight into whether it's due to explicit maps which aren't part of about:memory's "explicit" count, or due to something else.

 * js-gc-heap-chunk-unused  [ 1 /  46 / 13]
 * heap-unclassified        [18 /  38 / 36]
 * system compartment       [13 /  17 / 28]
 * js/stack                 [ 4 /   8 /  8]

 * total above              [36 / 109 / 85]
 * explicit - total         [ 8 /  13 / 15]

Since [explicit minus total] is approximately the same for the three measurements, the four reporters listed here are sufficient to explain most of the increase in explicit.

I'm not sure why the system compartment grew between the second and third measurements, but this doesn't phase me too much.  The growth is due to objects and shapes, so presumably it's useful data.

We're working on mitigating the js-gc-heap-chunk-unused increase, in bug 671702.

As for heap-unclassified...I think this lends credence to njn's idea that there's at least some JS memory in heap-unclassified.  But we'll have to see, I guess.

So a number of bugs we're working on should address this bug.  I'll set dependencies, and then we can forget about this one for a while.

	Nicholas Nethercote [inactive]
	Comment 15 • 13 years ago

> this doesn't phase me too much.

"faze"

	Justin Lebar (not reading bugmail)
	Comment 16 • 13 years ago

I redid this experiment with my smaps patch (bug 674290).  Here are the numbers [startup / after closing tabs] in mb:

* explicit            [59 / 102]
* heap-unclassified   [25 /  53]
* resident            [94 / 164]
* anonymous maps' RSS [58 / 127]
* other maps' RSS     [36 /  37]

"anonymous maps' RSS" is the amount of physical memory used by anonymous mmaps.  These are mostly jemalloc's, but other modules create them as well.

"other maps' RSS" is the amount of physical memory used by everything that's not an anonymous map.  This is mostly code.

Of note here:

 1) heap-unclassified increases by 25mb.  This means that some memory usage we're not tracking grew.  (heap-unclassified counts fragmentation internal to allocations -- we've called this "slop" or "clownshoes": we request X bytes but the allocator reserves Y > X bytes instead -- but it doesn't count fragmentation external to the allocation, where jemalloc holds onto X bytes from the OS for Y bytes of allocations, with X > Y.)

 2) the difference between anonymous maps' RSS and explicit grew from 0 to 25mb.

   This could be fragmentation in jemalloc, or this could be dark matter (we could be mmaping pages outside the heap allocator and not reporting it).  jemalloc reports low amounts of fragmentation (heap-dirty is ~3mb), but I'm not positive that's the right number for us to be looking at.

   I suspect that this is fragmentation in jemalloc, because I'd presume that if we created dark matter, some would have existed before we opened up the tabs as well.  But the difference between anonymous maps' RSS and explicit was originally 0.

   It might not be a coincidence that both of these numbers are 25mb, but I'm not sure what it would signify.

We should be able to use DMD (njn's Valgrind tool for finding dark matter) to address the heap-unclassified issue in (1).

I think the first step in addressing (2) should be getting a better understanding of jemalloc's memory statistics.  Then we'll either have confidence that heap-dirty is all the fragmentation to speak of, or try to improve jemalloc's reporting.

	Justin Lebar (not reading bugmail)
	Comment 17 • 13 years ago

So heap-dirty definitely isn't the right number.  Pages are dirty if they're not being used for anything, but if jemalloc hasn't madvise(FREE / DONTNEED)'ed them.

So if a page is half-full, it's not dirty.  Clearly heap-dirty doesn't count all the fragmentation in jemalloc.

When MALLOC_DECOMMIT is not defined, jemalloc uses madvise(FREE / DONTNEED) to give a page back to the OS.  But jemalloc still counts this page as "committed" because the page is writable -- it's just a zeroed, copy-on-write page which doesn't count against our RSS.  AFAICT, jemalloc doesn't keep statistics counting how many of these zeroed COW pages are sitting around.  So this makes it hard to know exactly how much RSS jemalloc is responsible for, which is what we're trying to find out here.

But with MALLOC_DECOMMIT defined, every page in heap-committed should, in theory, count against RSS.  That matches my observations:

 With MALLOC_DECOMMIT, [startup / after closing tabs] in mb

  heap-commmitted:     [46.6 / 114.1]
  anonymous maps' RSS: [46.4 / 114.4]

The fact that heap-committed corresponds almost exactly to the anonymous maps' RSS, both before and after closing the tabs, indicates to me that the increase in RSS here is due to fragmentation in jemalloc, not dark matter.

	Justin Lebar (not reading bugmail)
	Comment 18 • 13 years ago

I think there are three things to be fixed for this bug:

 * jemalloc's fragmentation - Maybe fixed by upgrading to the latest version of jemalloc.

 * increase in heap-unclassified - Fixed by DMD.

 * increase in explicit memory usage - bug 671702 may help here.

	Justin Lebar (not reading bugmail)
	Comment 19 • 13 years ago

Why does this block bug 659120?