Bug 765874 regressed the Trace Malloc Allocs and MaxHeap benchmarks on Windows and Linux. For example: Trace Malloc MaxHeap increase 2.31% on CentOS (x86_64) release 5 (Final) Mozilla-Inbound Previous: avg 49991676.667 stddev 355259.214 of 30 runs up to revision 30a4cd976842 New : avg 51145820.000 stddev 3334.217 of 5 runs since revision 0e5b85c4fdde Change : +1154143.333 (2.31% / z=3.249) Graph : http://mzl.la/MqLUPh Changeset range: http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=30a4cd976842&tochange=0e5b85c4fdde Requesting tracking-firefox16 because this is a memory usage regression from a new feature in Firefox 16.

Note: Bug 765874 was also in the regression range for several Ts (dirty) regressions on Linux platforms.

> Change : +1154143.333 This is in bytes? If so, this is a 1MB regression, which is notable, but not huge. But we should try to understand if it's a constant 1MB or if it increases with usage; that would affect our prioritization...

The code in bug 765874 does very little to affect the default behavior of the app: adds a bit of extra elements in browser.xul, adds some code to browser.js, most of it disabled by default. It does trigger the loading of several new JS modules, though: SocialService.jsm, SocialProvider.jsm, FrameWorker.jsm, WorkerAPI.jsm, and Social.jsm. They're all under 5K of JS, except for FrameWorker.jsm which is 19K. What's the typical allocs/maxheap impact of loading an additional JS module? (I bet no one knows that answer!) One thing we can try as an experiment is to lazy-load some of these modules, to avoid loading them in the current state, and see what impact that has.

This should prevent the loading of FrameWorker.jsm and WorkerAPI.jsm in a normal startup. It may also prevent the loading of SocialProvider.jsm, depending on how quickly the test shuts down the browser. It won't have any effect on a build that actually enables Social features, which we plan to do at some point.

> They're all under 5K of JS, except for FrameWorker.jsm which is 19K. I suspect code size is a poor indicator of memory consumptionf for a JSM. > What's the typical allocs/maxheap impact of loading an additional JS > module? (I bet no one knows that answer!) Here's the smallest JSM in my current Linux64 session, as reported by about:memory: │ │ │ ├──────67,064 B (00.03%) -- compartment([System Principal], resource://gre/modules/PrivateBrowsingUtils.jsm) │ │ │ │ ├──57,344 B (00.02%) -- gc-heap │ │ │ │ │ ├──40,984 B (00.02%) ── unused-gc-things │ │ │ │ │ └──16,360 B (00.01%) ── sundries │ │ │ │ └───9,720 B (00.00%) ── other-sundries There are quite a few in the 70KB -- 200KB range, and a handful that are bigger. resource:///modules/sessionstore/SessionStore.jsm is the biggest at 2.4MB. 1MB for these modules sounds higher than I'd expect, but plausible. Take a look at what about:memory says for these new modules.

(In reply to Nicholas Nethercote [:njn] from comment #5) > > They're all under 5K of JS, except for FrameWorker.jsm which is 19K. > > I suspect code size is a poor indicator of memory consumptionf for a JSM. Apart from SocialService/SocialProvider/Social, the code loaded in these modules isn't run, which is why I mentioned their size. Looking at about:memory in my Mac opt build, for the 5 modules mentioned in comment 3, I see: │ │ │ ├──────90,920 B (00.13%) -- compartment([System Principal], resource://gre/modules/SocialService.jsm) │ │ │ │ ├──73,728 B (00.10%) -- gc-heap │ │ │ │ │ ├──45,624 B (00.06%) ── unused-gc-things │ │ │ │ │ ├──18,600 B (00.03%) ── sundries │ │ │ │ │ └───9,504 B (00.01%) ── objects/function │ │ │ │ └──17,192 B (00.02%) ── other-sundries │ │ │ ├──────78,424 B (00.11%) -- compartment([System Principal], resource://gre/modules/SocialProvider.jsm) │ │ │ │ ├──65,536 B (00.09%) -- gc-heap │ │ │ │ │ ├──42,760 B (00.06%) ── unused-gc-things │ │ │ │ │ └──22,776 B (00.03%) ── sundries │ │ │ │ └──12,888 B (00.02%) ── other-sundries │ │ │ ├──────77,464 B (00.11%) -- compartment([System Principal], resource://gre/modules/WorkerAPI.jsm) │ │ │ │ ├──65,536 B (00.09%) -- gc-heap │ │ │ │ │ ├──41,448 B (00.06%) ── unused-gc-things │ │ │ │ │ └──24,088 B (00.03%) ── sundries │ │ │ │ └──11,928 B (00.02%) ── other-sundries │ │ │ ├──────97,528 B (00.13%) -- compartment([System Principal], resource://gre/modules/FrameWorker.jsm) │ │ │ │ ├──73,728 B (00.10%) -- gc-heap │ │ │ │ │ ├──39,488 B (00.05%) ── unused-gc-things │ │ │ │ │ ├──22,752 B (00.03%) ── sundries │ │ │ │ │ └──11,488 B (00.02%) ── objects/function │ │ │ │ ├──12,112 B (00.02%) ── other-sundries │ │ │ │ └──11,688 B (00.02%) ── script-data │ │ │ ├─────213,768 B (00.29%) -- compartment([System Principal], resource:///modules/Social.jsm) │ │ │ │ ├──131,072 B (00.18%) ── analysis-temporary │ │ │ │ ├───69,632 B (00.10%) -- gc-heap │ │ │ │ │ ├──45,560 B (00.06%) ── unused-gc-things │ │ │ │ │ ├──15,688 B (00.02%) ── sundries │ │ │ │ │ └───8,384 B (00.01%) ── objects/function │ │ │ │ └───13,064 B (00.02%) ── other-sundries All together that's ~545K. The "analysis-temporary" piece of Social.jsm looked sketchy (more than have of its size). MXR suggests it's TI related, and should be "Cleared on GC", but I don't see that happening after having let my browsing sit for a while. Hitting the "minimize memory usage" button does kill it, though, turning Social.jsm into: │ │ │ ├──────78,536 B (00.12%) -- compartment([System Principal], resource:///modules/Social.jsm) │ │ │ │ ├──65,536 B (00.10%) -- gc-heap │ │ │ │ │ ├──43,216 B (00.06%) ── unused-gc-things │ │ │ │ │ └──22,320 B (00.03%) ── sundries │ │ │ │ └──13,000 B (00.02%) ── other-sundries

(In reply to :Gavin Sharp (use gavin@gavinsharp.com for email) from comment #6) > All together that's ~545K. My debug build has roughly equivalent numbers. > The "analysis-temporary" piece of Social.jsm looked sketchy (more than half of its > size). I've noticed the same behavior for SocialService.jsm, though its "analysis-temporary" bits disappear on their own eventually (after refreshing about:memory). Assuming this pattern is consistent (analysis-temporary roughly doubling a module's footprint as it's loaded), it could explain a ~1MB MaxHeap regression. The modules in question don't appear to be doing anything stupid, memory-wise, and are among the smallest listed in about:memory. This looks like a case of us just adding a bunch of code, and having the minimum possible overhead from those additions add up over several modules (which all started getting loaded by default on startup when bug 765874 landed). We can land the lazy loading patch to reduce this impact temporarily, but one day we'll enable all of this code anyways (or our users will in the wild, despite it not being covered by the MaxHeap workload). I suspect there isn't much we can do here.

I landed the lazy-loading patch to see what affect it has. I'll followup and with some local testing to confirm. https://hg.mozilla.org/integration/mozilla-inbound/rev/bc090e21a088

Push backed out for mochitest-a11y failures: https://tbpl.mozilla.org/php/getParsedLog.php?id=13567358&tree=Mozilla-Inbound { 20840 ERROR TEST-UNEXPECTED-FAIL | chrome://mochitests/content/a11y/accessible/tree/test_dochierarchy.html | Wrong child document count of root accessible - got 2, expected 1 } https://hg.mozilla.org/integration/mozilla-inbound/rev/082542b01af8

Re-landed, since it wasn't the cause of the bustage: https://hg.mozilla.org/mozilla-central/rev/6fdf9985acfe

I ran some MaxHeap tests locally on Linux (enabled trace malloc, used leaktest.py -- --trace-malloc malloc.log for 2-3 runs, and then ran leakstats on malloc.log). Here are the results: Social modules disabled entirely: Maximum Heap Size: 50815457 bytes Maximum Heap Size: 50820146 bytes Maximum Heap Size: 50820545 bytes FrameWorker/WorkerAPI lazy load: Maximum Heap Size: 51357732 bytes Maximum Heap Size: 51360681 bytes No lazy load, all modules loaded: Maximum Heap Size: 51621253 bytes Maximum Heap Size: 51709321 bytes That shows a 868KB regression in MaxHeap from all of the social modules. Lazy loading WorkerAPI and FrameWorker reduces the regression by 340KB. We can merge SocialService and SocialProvider to eliminate some module overhead, I filed bug 774543 for that. I don't think there are any other such merging opportunities that make sense - the other modules are really functionally separate and combining them just to shave off a few KB of maxheap doesn't seem worth it.

Sounds reasonable to me. The real fix here is to win back the CPG fragmentation losses (bug 764220). Thanks for investigating, Gavin.

Going to call this WONTFIX, even though the patch here did reduce the regression somewhat.