Open Bug 1046169 Opened 10 years ago Updated 2 years ago

unusually high heap-overhead/bin-unused after ~1 week of light browsing

Categories

(Core :: General, defect)

x86_64
Linux
defect

Tracking

()

People

(Reporter: froydnj, Unassigned)

References

Details

(Whiteboard: [MemShrink:P2])

Attachments

(1 file)

Attached file memory report (deleted) —
I have a separate browser profile for my online todo list and for private-browsing gmail. I've had a session in this profile open for about a week, browsing a few other sites along with my normal todo list and mail usage. Imagine my surprise when I measured with about:memory and found: 465.26 MB (100.0%) -- explicit ├──152.73 MB (32.83%) -- heap-overhead │ ├──141.44 MB (30.40%) ── bin-unused │ ├────8.49 MB (01.82%) ── bookkeeping │ └────2.80 MB (00.60%) ++ (2 tiny) STR are obviously long and not really guaranteed to R, but logging this here in case somebody has bright ideas about what's going wrong. Maybe bin-unused needs more detail? bkelly suggested bug 1005844 might have helped here, too.
(In reply to Nathan Froyd (:froydnj) from comment #0) > 465.26 MB (100.0%) -- explicit > ├──152.73 MB (32.83%) -- heap-overhead > │ ├──141.44 MB (30.40%) ── bin-unused This is actually a good thing in a way, out of the overhead the majority is memory that _can_ be used, just hasn't been used yet. > │ ├────8.49 MB (01.82%) ── bookkeeping This is even better, bookkeeping below 2% is a good sign that we're not wasting a lot of overhead on jemalloc internals. > Maybe bin-unused needs more detail? It's about as detailed as it's going to get unfortunately -- we could split out into each bin size, but I don't think that would be useful. That indicates there are a bunch of partially filled runs hanging around (internal fragmentation). Basically a bunch of memory was used at one point, less is used now, but the runs that were allocated to service the higher memory usage aren't emptied out yet and can't be recycled/freed. I'm guessing this is a machine with a lot of RAM where there probably isn't memory pressure, what happens if you do a minimize and then report? > bkelly suggested bug 1005844 might have helped here, too. If I recall correctly, that's a Windows only change and has to do with virtual address space fragmentation.
(In reply to Eric Rahm [:erahm] from comment #1) > It's about as detailed as it's going to get unfortunately -- we could split > out into each bin size, but I don't think that would be useful. I was wondering if the bin sizes might help guide us to poking around in C++ areas (smaller bins) or JS areas (larger bins), but perhaps that's not helpful. > I'm guessing this is a machine with a lot of RAM where there probably isn't > memory pressure, what happens if you do a minimize and then report? The machine has 8GB, x86-64 Linux. This report was after a minimize, sadly enough.
I'd like to say that this bug is one of the most important problems of Firefox. I thought Compacting GC was about this, but have been told now that is about unused-gc-things. unused-gc-things is not a problem for me, but heap-overhead is huge after browsing a lot and closing (almost) everything. Bug 668809 should be duped to this one, and not Compacting GC, in my opinion. On AWSY, I think is possible to say that difference between the light green dot and the purple one is from this bug. Even more thinking about the "regression" caused by Bug 898558. On my daily browsing, I get around 100mb of bin-unused. Just check some emails on Gmail and browse a few minutes on Facebook, logout from both sites, check about:memory and you'll get this.
Whiteboard: [MemShrink] → [MemShrink:P2]
Nathan can you do a test run with jemalloc 3 to get see if we get the same behavior? I went back and looked at our compatibility layer and I believe the reporting will be a bit different ('bin-unused' will be 0, and that value will be included in 'waste' instead), but the heap-overhead number should be comparable. To do a jemalloc 3 build all you need to do is add the following to your mozconfig: MOZ_JEMALLOC3=1 It's possible this is a regression of sorts from bug 1006769, at the time we knew that RSS might go up but there was an overall win on b2g (I'd suggest reading through the bug to get a better feel for what happened). The tradeoff is essentially a higher amount of bookkeeping vs a higher amount of bin-unused, we chose lower bookkeeping. Bookkeeping is essentially waste, bin-unused has potential to be used.
Flags: needinfo?(nfroyd)
(In reply to Eric Rahm [:erahm] from comment #4) > Nathan can you do a test run with jemalloc 3 to get see if we get the same > behavior? I went back and looked at our compatibility layer and I believe > the reporting will be a bit different ('bin-unused' will be 0, and that > value will be included in 'waste' instead), but the heap-overhead number > should be comparable. > > To do a jemalloc 3 build all you need to do is add the following to your > mozconfig: > MOZ_JEMALLOC3=1 I'll try this and report back sometime next week.
Flags: needinfo?(nfroyd)
Hello, I've been experiencing this problem too, after closing some tabs (facebook, gmail) I have very high numbers for bin-unused memory (340mb). For what I see in this thread, it appears this memory can be reused, but what if I was navigating through some sites, then noticed high memory usage and decided to close some tabs to free some memory? Right now, most of the freed memory goes into bin-unused, which forces me to restart firefox in order to actually clear the memory used by it. Why is memory put on bin-unused in the first place? I haven't been able to find much information about this issue, any help/guidance will be much appreciated.
(In reply to Nathan Froyd [:froydnj] [:nfroyd] from comment #5) > (In reply to Eric Rahm [:erahm] from comment #4) > > Nathan can you do a test run with jemalloc 3 to get see if we get the same > > behavior? I went back and looked at our compatibility layer and I believe > > the reporting will be a bit different ('bin-unused' will be 0, and that > > value will be included in 'waste' instead), but the heap-overhead number > > should be comparable. > > > > To do a jemalloc 3 build all you need to do is add the following to your > > mozconfig: > > MOZ_JEMALLOC3=1 > > I'll try this and report back sometime next week.
Flags: needinfo?(nfroyd)
Still haven't looked at this consistently, will put it on the todo list.
Flags: needinfo?(nfroyd)
I just randomly checked about:memory on my Windows 10 desktop after Nightly had been running idle for a while and found a bunch of "unused" memory, searched for "bin-unused," and found this bug: 565.21 MB (100.0%) -- explicit ├──163.24 MB (28.88%) -- js-non-window ├──147.85 MB (26.16%) -- heap-overhead │ ├──135.37 MB (23.95%) ── bin-unused │ ├────9.80 MB (01.73%) ── bookkeeping │ └────2.68 MB (00.47%) ++ (2 tiny) ├──120.12 MB (21.25%) -- window-objects ├───65.57 MB (11.60%) ── heap-unclassified 23.95% of memory allocated to Firefox just sitting around idle seems like a waste of memory :/
1,469.87 MB (100.0%) -- explicit ├────589.83 MB (40.13%) -- heap-overhead │ ├──553.27 MB (37.64%) ── bin-unused │ ├───32.80 MB (02.23%) ── bookkeeping This is in 40 on OSX *after* closing all (400+ tabs) (except 8 pinned tabs), and doing gc + cc. Activity Monitor says Firefox is still using 2.45 GB (it was 7+ GB before, with frequent multi-second beachballs, which seems to be new for me as of 40?)
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: