User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:9.0a1) Gecko/20110829 Firefox/9.0a1 Build ID: 20110829055008 Steps to reproduce: I created a simple php to create some huge tables http://mrecho.net/firefox/tables.phps I made sure I disabled everything, only thing that loaded at start up was about:memory. On disk the html is 27 MB, so that is 42 times the size of on disk file! No css or js on page. On a good note, when the tab is closed, it goes back to about the same state with about:memory, but the resident hangs around 900+MB. Built from http://hg.mozilla.org/mozilla-central/rev/33031c875984 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:9.0a1) Gecko/20110829 Firefox/9.0a1 Actual results: Loaded my tables.php, and 1,148MB of ram used. 1,148.00 MB (100.0%) -- explicit 513.15 MB (44.70%) -- layout 316.73 MB (27.59%) -- heap-unclassified 287.58 MB (25.05%) -- dom 24.14 MB (02.10%) -- js This is after hitting "minimize memory usage" 10+ times.

Theres 2 goals of this post. 1: Memory allocation in different parts of the system for simple text on screen. 2: Hope to help out on finding heap allocation and fragmentation issues. Back in the late 90's I was able to load tables like this, yes they would take awhile to load, but the system(windows) only had maybe 64-128MB of ram.

This is only really a problem if we're substantially worse than other browsers. But it is quite likely that we are worse -- http://www.neowin.net/forum/topic/989780-meet-firefox-next/page__st__720__p__594231040#entry594231040 showed we're much worse on a different large page. But we need measurements from other browsers to get an idea of what the end goal of this bug is. Trevor, if you could get RSS measurements (from 'ps' or 'top' on Mac) for other browsers that would be very helpful.

VSZ RSS e6591ea9b27b FF Nightly(safemode) 4,668,888 1,446,152 5.1.7534.48.3 Safari WebProcess 4,907,288 1,346,612 13.0.782.218 Chrome helper 1,544,596 757,756 11.50.1074 Opera 1,292,704 472,220

Thanks for the info, Trevor. I think VSZ is pretty meaningless on Mac but the RSS numbers are clear. The neowin.net measurements also showed that Opera was the best.

I wish I could help more, but im not a c/c++ guy. But with my experience of Gentoo, I know how to compile from source, and run gdb/strace. I do have FF built from current source on the mac if you want me to run more tests.

nsBlockReflowContext: Block(body)(2)@0x107478f68 metrics=84840,65410560! nsBlockReflowContext: Block(body)(2)@0x107478f68 metrics=84840,115013328! nsBlockReflowContext: Block(body)(2)@0x107478f68 metrics=84840,126572784! nsBlockReflowContext: Block(body)(2)@0x107478f68 metrics=84840,136532880! nsBlockReflowContext: Block(body)(2)@0x107478f68 metrics=84840,146494608! nsBlockReflowContext: Block(body)(2)@0x107478f68 metrics=84840,156454704! nsBlockReflowContext: Block(body)(2)@0x107478f68 metrics=84840,163526400!

Linux Mozilla/5.0 (X11; Linux i686; rv:9.0a1) Gecko/20110831 Firefox/9.0a1 ID:20110831030834 : 627.41 MB (100.0%) -- explicit ├──336.46 MB (53.63%) -- layout │ ├──336.13 MB (53.57%) -- arenas │ └────0.33 MB (00.05%) -- (1 omitted) ├──155.22 MB (24.74%) -- dom ├──111.38 MB (17.75%) -- heap-unclassified ├───19.54 MB (03.11%) -- js Just saving the page to disk increased mem usage: 992.58 MB (100.0%) -- explicit ├──479.52 MB (48.31%) -- heap-unclassified ├──336.56 MB (33.91%) -- layout │ ├──336.24 MB (33.88%) -- arenas │ └────0.33 MB (00.03%) -- (1 omitted) ├──155.22 MB (15.64%) -- dom ├───15.99 MB (01.61%) -- js Opera 11.50 on exactly the same saved HTML: RSS 455m

The size difference here is worth investigating. Based on the numbers it looks like we just have a ton of frames or other layout objects.

So this testcase is producing 100 tables, each with one thead and one row in the thead, then a tbody and 1000 rows in the tbody, right? And each row has 10 cells, and each wife has seven sacks and each sack has seven cats... er. In any case, a total of 100*(1000+1)*10 = 1001000 table cells, 100100 table rows, 200 table row groups, and 100 tables. Each table cell involves an nsTableCellFrame and an nsBlockFrame, as well as a cellmap entry. Plus all the text in the cells. In a 32-bit build, nsBlockFrame is 108 bytes; nsTableCellFrame is 92 bytes. Cellmap entries are 4 bytes in the presshell arena and another 4 or so in the actual cellmap array storage, so another 8 bytes per cell ignoring slop issues. A textframe is 72 bytes. So just those things account for about 280MB of the frame arena usage. That's out of 330MB total reported in comment 7. In terms of the DOM, nsHTMLTableCellElement is 68 bytes in a 32-bit build, so that's another 70MB of memory right there. nsTextNode is another 64 bytes, so that's another 65MB. Again, this is most of the DOM memory reported in comment 7. In a 64-bit build, the numbers will be bigger, of course. Almost certainly by about 2x. Note that the numbers in comment 3 are comparing 2 64-bit browsers (us and Safari) and 2 32-bit ones (Chrome and Opera). I'd certainly expect the latter to use less memory!

Is there a page somewhere the lays out the whole data structure / tree of a page? All the way from the window border to the <td> ? One huge map of all the arrays and trees. nsTableCellFrame -> nsBlockFrame -> text? dom -> nsHTMLTableCellElement -> nsTextNode -> text? Is there any round up of allocations going on with all these different byte sizes? And may contribute to the heap-unclassified? Some of these byte counts sound really huge. The GPS stuff I do at work is 1/2 the amount of bytes, NEMA binary.

> Is there a page somewhere the lays out the whole data structure / tree of a page? There is not. You can see what the DOM looks like using utilities like DOM Inspector, Firebug, or even Live DOM Viewer. In a debug build, you can dump out the frame tree and style context tree. But that won't tell you about various auxiliary data structures, of course. > Is there any round up of allocations going on with all these different byte sizes? It depends. Frame allocations come out of arenas, so the only rounding is that you lose (arena size) % (frame size) bytes per arena, plus whatever part of the last arena doesn't fill up for your allocation count. In this case, arena size is 8192 bytes, so the overhead there is no more than about 100 bytes per 8KB. For allocations via malloc(), slop can in fact contribute to heap-unclassified; njn has been measuring that sort of thing. Speaking of which, running DMD on this page might be enlightening. > Some of these byte counts sound really huge. The per-object ones or the total ones? ;)

So does nsTableCellFrame + nsBlockFrame + text go into a single arena, and once thats full, it starts to use another one? Im just curious if we use all 8K bytes before allocating the next 8K. Just not really sure what an Arena consists of. Also it sounds like there are 3 different things going on with this table data, Frames, DOM, and Cellmap. I assume there all different types of memory allocations. I was talking about nsBlockFrame being 108 bytes, just sounds like a lot of data. And is that information being duplicated in nsTableCellFrame or any other Frame? Sorry, im just an ideas / questioning guy, I try to provoke thoughts about stuff ;)

As soon as an allocation request can't be satisfied from the current 8KB arena, we allocate a new one, and the remaining bytes in the first one are never used.

Each frame type comes out of a different set of arenas. So the nsTableCellFrame, the nsBlockFrame, and the nsTextFrame all come out of different arenas. But for each frame type we wait for the arena to be full before creating a new one. > there are 3 different things going on with this table data, Frames, DOM, and Cellmap. Yes. > I assume there all different types of memory allocations. Somewhat. DOM nodes are just allocated via operator new. Frames are arena-allocated. Cellmap entries are also arena-allocated, but there's an array of arrays of pointers to them, and the array buffers are just malloced. > I was talking about nsBlockFrame being 108 bytes, just sounds like a lot of data. Sure. For what it's worth, that breaks down as follows: vtable pointer: 4 rect (top/left/width/height): 16 pointer to DOM node: 4 pointer to style context: 4 pointer to parent frame: 4 pointer to previous sibling: 4 pointer to next sibling: 4 state bits: 8 overflow information: 4 pointer to previous continuation: 4 pointer to next continuation: 4 first child: 4 last child: 4 cached minimum width: 4 cached preferred width: 4 first line box pointer: 4 last line box pointer: 4 first floated child: 4 last floated child: 4 pointer to bullet: 4 pointer to first abs pos child: 4 pointer to last abs pos child: 4 pointer to name of abs pos child list: 4 These last 12 bytes are likely to go away once ehsan refactors how we do absolutely positioned kids. > And is that information being duplicated in nsTableCellFrame or any other Frame? No.

1,398,517,112 B (100.0%) -- explicit ├────538,154,792 B (38.48%) -- layout │ ├──536,961,752 B (38.40%) -- shell(tables.php.html) │ │ ├──536,887,944 B (38.39%) -- arenas │ │ └───────73,808 B (00.01%) -- styledata ├────526,970,479 B (37.68%) -- heap-unclassified ├────301,533,429 B (21.56%) -- dom ├─────23,133,594 B (01.65%) -- js d772dfb96ba1 safe-mode from the cli Is there a way to force 32bit mode from the cli?

arch -i386 /path/to/firefox

32Bit mode, usage is smaller, but not 1/2. 799,864,920 B (100.0%) -- explicit ├──352,720,512 B (44.10%) -- layout │ ├──351,997,572 B (44.01%) -- shell(file:///Users/mrecho/Desktop/tables.php.html) │ │ ├──351,958,392 B (44.00%) -- arenas │ │ └───────39,180 B (00.00%) -- styledata ├──259,427,074 B (32.43%) -- heap-unclassified ├──162,703,301 B (20.34%) -- dom ├───17,363,159 B (02.17%) -- js

OSX x86_64 b57d50c6c046 Status update, after hitting minimize a few times... 1,104,878,368 B (100.0%) -- explicit ├────538,206,224 B (48.71%) -- layout │ ├──536,960,856 B (48.60%) -- shell(tables.php.html) │ │ ├──536,886,952 B (48.59%) -- arenas │ │ └───────73,904 B (00.01%) -- styledata ├────301,532,845 B (27.29%) -- dom ├────233,315,040 B (21.12%) -- heap-unclassified ├─────23,209,024 B (02.10%) -- js │ ├──────88,608 B (00.01%) -- compartment(tables.php.html) │ │ ├──77,824 B (00.01%) -- gc-heap │ │ │ ├──34,192 B (00.00%) -- arena-unused │ │ │ ├──22,720 B (00.00%) -- objects │ │ │ ├──17,664 B (00.00%) -- shapes │ │ │ ├───1,184 B (00.00%) -- arena-padding │ │ │ ├───1,024 B (00.00%) -- type-objects │ │ │ ├─────608 B (00.00%) -- arena-headers │ │ │ └─────432 B (00.00%) -- scripts │ │ ├───6,144 B (00.00%) -- object-slots │ │ ├───1,472 B (00.00%) -- property-tables │ │ ├───1,152 B (00.00%) -- shape-kids │ │ ├───1,024 B (00.00%) -- type-inference │ │ │ └──1,024 B (00.00%) -- object-main │ │ ├─────960 B (00.00%) -- object-empty-shapes │ │ └──────32 B (00.00%) -- analysis-temporary

OSX 10.7.2 x86_64 rev/04b4ea333800 21,939,504 B (100.0%) -- explicit ├──522,557,384 B (2381.81%) -- layout │ ├──520,972,208 B (2374.59%) -- shell(tables.php.html) ├──334,171,529 B (1523.15%) -- dom ├──24,031,584 B (109.54%) -- js └──-869,789,110 B (-3964.49%) -- heap-unclassified 235,280 B -- gfx-surface-image 0 B -- heap-allocated 4,096 B -- heap-committed 100.00% -- heap-committed-unallocated-fraction 0 B -- heap-dirty 1,048,576 B -- heap-unallocated 2 -- js-compartments-system 2 -- js-compartments-user 11,534,336 B -- js-gc-heap 696,944 B -- js-gc-heap-arena-unused 0 B -- js-gc-heap-chunk-clean-unused 3,231,744 B -- js-gc-heap-chunk-dirty-unused 34.06% -- js-gc-heap-unused-fraction 34,880 B -- js-total-analysis-temporary 65,536 B -- js-total-mjit 3,347,896 B -- js-total-objects 2,162,080 B -- js-total-scripts 2,690,240 B -- js-total-shapes 1,406,912 B -- js-total-strings 174,944 B -- js-total-type-inference 17 -- page-faults-hard 556,252 -- page-faults-soft 1,533,952,000 B -- resident 5,043,568,640 B -- vsize

That "explicit" number is completely bogus (and hence so is heap-unclassified). Please file a separate bug on that?

Ive noticed that its been a negative for awhile now. I would have assumed that someone would have already filed a bug.

There's been no bug filed that I've seen (and I watch all incoming core bugs). Chances are, the issue is specific to OSX 10.7 if not more narrowly than that... Please do file a bug, ok?

I filed bug 698953 for the bogus about:memory values.

rev/c2102c45c8da 1,071.04 MB (100.0%) -- explicit ├────498.11 MB (46.51%) -- layout │ ├──496.85 MB (46.39%) -- shell(file:///tables.php.html) │ │ ├──496.72 MB (46.38%) -- arenas │ │ └────0.13 MB (00.01%) -- (1 omitted) │ └────1.27 MB (00.12%) -- (3 omitted) ├────318.69 MB (29.76%) -- dom ├────225.89 MB (21.09%) -- heap-unclassified ├─────21.79 MB (02.03%) -- js 0.22 MB -- gfx-surface-image 1,052.49 MB -- heap-allocated 1,073.70 MB -- heap-committed 1.97% -- heap-committed-fragmentation 3.14 MB -- heap-dirty 157.51 MB -- heap-unallocated 2 -- js-compartments-system 2 -- js-compartments-user 9.00 MB -- js-gc-heap 1.38 MB -- js-gc-heap-arena-unused 0.00 MB -- js-gc-heap-chunk-clean-unused 0.37 MB -- js-gc-heap-chunk-dirty-unused 2.07 MB -- js-gc-heap-decommitted 42.44% -- js-gc-heap-unused-fraction 0.13 MB -- js-total-analysis-temporary 0.00 MB -- js-total-mjit 1.97 MB -- js-total-objects 2.29 MB -- js-total-scripts 2.53 MB -- js-total-shapes 1.35 MB -- js-total-strings 0.07 MB -- js-total-type-inference 10 -- page-faults-hard 588,913 -- page-faults-soft 1,145.60 MB -- resident 4,593.08 MB -- vsize

Just wanted to see where its at now a few years later. OSX 10.10.1 Build 08e3c773c747 33.1 Still building with MacOSX10.6.sdk ? 1,170.57 MB (100.0%) -- explicit ├────962.74 MB (82.25%) -- window-objects │ ├──950.19 MB (81.17%) -- top(file:///Users/mrecho/Desktop/table_hell.html, id=9) │ │ ├──950.11 MB (81.17%) -- active/window(file:///Users/mrecho/Desktop/table_hell.html) │ │ │ ├──572.61 MB (48.92%) -- layout │ │ │ │ ├──418.75 MB (35.77%) -- frames │ │ │ │ │ ├──145.10 MB (12.40%) ── nsBlockFrame │ │ │ │ │ ├──137.47 MB (11.74%) ── nsTableCellFrame │ │ │ │ │ ├──106.93 MB (09.13%) ── nsTextFrame │ │ │ │ │ ├───29.02 MB (02.48%) ── nsTableRowFrame │ │ │ │ │ └────0.23 MB (00.02%) ++ (6 tiny) │ │ │ │ ├───76.39 MB (06.53%) ── line-boxes │ │ │ │ ├───64.00 MB (05.47%) ── pres-contexts │ │ │ │ ├───13.33 MB (01.14%) ── pres-shell │ │ │ │ └────0.14 MB (00.01%) ++ (3 tiny) │ │ │ ├──377.40 MB (32.24%) -- dom │ │ │ │ ├──239.92 MB (20.50%) ── element-nodes │ │ │ │ ├──137.46 MB (11.74%) ── text-nodes │ │ │ │ └────0.01 MB (00.00%) ++ (2 tiny) │ │ │ └────0.11 MB (00.01%) ++ (3 tiny) │ │ └────0.07 MB (00.01%) ++ js-zone(0x118c98800) │ └───12.55 MB (01.07%) ++ (6 tiny) ├────130.31 MB (11.13%) ── heap-unclassified ├─────31.15 MB (02.66%) -- js-non-window │ ├──22.42 MB (01.92%) -- zones │ │ ├──20.52 MB (01.75%) ++ zone(0x10038e800) │ │ └───1.90 MB (00.16%) ++ (2 tiny) │ └───8.73 MB (00.75%) ++ (2 tiny) ├─────27.82 MB (02.38%) -- heap-overhead │ ├──18.01 MB (01.54%) ── bookkeeping │ └───9.80 MB (00.84%) ++ (3 tiny) └─────18.56 MB (01.59%) ++ (21 tiny)