Closed Bug 610064 Opened 14 years ago Closed 14 years ago

nsZipWriter::Open allocates a 3MB block, of which 98% is unused

Categories

(Core Graveyard :: File Handling, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 610040

People

(Reporter: jseward, Assigned: taras.mozilla)

Details

For a run of M-C of 29 Oct, x86_64-Linux, startup with 5 tabs with fairly complex pages, a bit of browsing, the heap maxes out at 72,395,085 bytes in 411,549 blocks. When ranking heap allocation stacks by maximum liveness, nsZipWriter::Open comes 6th from top, requiring 3,145,728 bytes, that is, about 4.3% of the maximum heap residency. The block is only held live for a few hundred thousand or a few million instructions (I've seen a couple of different instances of this) but nevertheless will increase the total demand for address space if this happens at any time when the residency is within 3MB of peak residency. Worse, in this case the allocated block is hardly used. According to the acc-ratios, only 2% of the bytes in it were actually touched. raw data: ======== SUMMARY STATISTICS ======== guest_insns: 75,213,429,144 max_live: 72,395,085 in 411,549 blocks tot_alloc: 2,035,850,111 in 6,282,282 blocks insns per allocated byte: 36 [...] -------------------- 6 of 5000 -------------------- max-live: 3,145,728 in 1 blocks tot-alloc: 6,291,456 in 2 blocks (avg size 3145728.00) deaths: 2, at avg age 287,401 (0.00% of prog lifetime) acc-ratios: 0.02 rd, 0.02 wr (143,682 b-read, 143,900 b-written) at 0x4C275B8: malloc (/home/sewardj/VgTRUNK/trunk/coregrind/m_replacemalloc/vg_replace_malloc.c:236) by 0x73ED15B: moz_xmalloc (memory/mozalloc/mozalloc.cpp:98) by 0x562A435: nsBufferedStream::Init(nsISupports*, unsigned int) (ff-opt/netwerk/base/src/../../../dist/include/mozilla/mozalloc.h:238) by 0x6276A13: nsZipWriter::Open(nsIFile*, int) (ff-opt/modules/libjar/zipwriter/src/../../../../dist/include/nsNetUtil.h:1096) by 0x570D682: mozilla::scache::StartupCache::PutBuffer(char const*, char const*, unsigned int) (startupcache/StartupCache.cpp:289)
This appears to be the result of hg diff -c53815 ./modules/libjar/zipwriter/src/nsZipWriter.cpp - rv = NS_NewBufferedOutputStream(getter_AddRefs(mStream), stream, 0x800); + rv = NS_NewBufferedOutputStream(getter_AddRefs(mStream), stream, 3 * 1024 * 1024); iow, the size of the stream buffer is increased from 2kB to 3MB. I can see that 2kB is a bit on the small size, but a factor of 1500 increase seems to me to be hugely excessive. Wouldn't changing it to (eg) 64kB be an ok compromise? This would presumably decrease the per-buffer-fill costs by a factor of 32 on average without using much more memory.
Assignee: nobody → tglek
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → DUPLICATE
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.