In populateThisMonth() we seem to load all the pings contents from disk: https://github.com/mozilla/fhr-jelly/blob/master/js/data_v4.js#L302 But processPing() only evaluates the contents of "main" and "crash" pings: https://github.com/mozilla/fhr-jelly/blob/master/js/data_v4.js#L201 We should fix this to only load the contents of those ping types, otherwise busy profiles with e.g. many test-pilot pings (seen in the wild: >13000) take a long time loading about:healthreport.

I've tested this manually on my machine by pointing my Firefox (release) to the local webserver. * I've submitted a custom test ping and it got archived. * It shows up in about:telemetry -> Raw data * It doesn't show up in the startup time plot (that gets generated by parsing the files)

Madalin, flagging you as per email thread. The contents of about:healthreport are actually hosted remotely. The updated version is already on our staging server and can be viewed by changing the pref "datareporting.healthreport.about.reportUrl" to https://fhr-dev.allizom.org/%LOCALE%/v4/ This doesn't require any API change on the client, so you can test it using the most recent client version (Release or Nightly). To submit (and archive) pings of a 'test-ping' type: 1) Go to about:telemetry 2) Open the web developer tools 3) Submit a ping by writing in the console: Cu.import("resource://gre/modules/TelemetryController.jsm", {}).TelemetryController.submitExternalPing('test-type', {}); We should check that: * the sites load properly * the displayed values make sense and match the local profile history and settings (e.g. displayed update setting is correct, click-to-play plugin count correct, ...) * the startup times shown in the graph should make sense (i.e. not be astronomical and correlate to actual started sessions on those dates) * the raw data display works (and shows the pings of the 'test-type' too) * Having ~>13000 pings doesn't lock up the page for seconds, as it did before we changed this page.

Verified in FF 51.0.1 and Nightly 54.0a1 (2017-02-02) The site loads properly and the all the information is displayed correctly (add-ons, plugin count, the startup times are looking good). I have created a script to generate 13000 ping and the first time I open the Raw Data page it takes approximately 1 minute to load as you can see in this screencast: https://www.screencast.com/t/2jxTfeoKI I've tried with both clean and used older profiles to make sure the results are accurate. In both cases the page loads in same approximate time. This is happening in both release and nightly. I have made the pref change so this was tested against the staging server ("datareporting.healthreport.about.reportUrl" to https://fhr-dev.allizom.org/%LOCALE%/v4/)

This is a profiling trace of loading about:healthreport with 13000 custom pings in my VM. It uses the current staged files (no document fragments).

This is still loading 13k pings in about:healthreport, but the page was slightly modified and it's using document fragments instead of appending directly to the DOM.

This is a bit tricky. In my VM, the page with 13k pings loads in a few seconds, with or without the changes in the new PR. However, digging into the numbers from the DevTools Performance tool, it seems that |populateThisMonth| takes up a consistent fraction of the call time (~4.3%). Using the document fragment patch allows to drive down this number to about 1.9%. Other numbers go down as well. I've attached both profiler traces, they can be opened with the devtools (it's particularly interesting to see Call Tree and the Flame Chart).

Madalin, could you try this again? On a few hours, so that [1] gets on the staging server. It should load faster. [1] - https://github.com/mozilla/fhr-jelly/pull/180

I have repeated the tests in stage and it feels faster. I have performed multiple tests and the times are similar like the one in this screencast: https://www.screencast.com/t/ysmGIEfEoH

Thank you Madalin. It looks like this has received enough QA attention ;-) Let's close this and deploy the fixes in the dependant bug.