Closed
Bug 1176050
Opened 9 years ago
Closed 5 years ago
Firefox liveload with lots of connections slow when netmonitor is open
Categories
(DevTools :: Netmonitor, defect, P2)
Tracking
(Performance Impact:medium)
RESOLVED
FIXED
Performance Impact | medium |
People
(Reporter: petcuandrei, Unassigned)
References
(Blocks 2 open bugs)
Details
(Keywords: DevAdvocacy, perf:pageload, Whiteboard: [polish-backlog][difficulty=medium][DevRel:P2])
Attachments
(2 files)
(deleted),
text/plain
|
Details | |
(deleted),
patch
|
Details | Diff | Splinter Review |
User Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:38.0) Gecko/20100101 Firefox/38.0 Build ID: 20150605133456 Steps to reproduce: 1) install this js app from web git clone git@github.com:andreicristianpetcu/foobar.git cd foobar git checkout ef75f84 npm install -g grunt npm install bower install grunt server go to http://localhost:8000 and either run in the cli "touch app/index.html" or click the first link in the page several times. Actual results: The DevTools network tab shows the connection is really slow, it loads the whole page in 4 seconds while Chromium loads the page in 532 milliseconds. Firefox Nightly loads it a little faster than the stable version in around 3.5 seconds (probably due to bug 1143224 being fixed). I think this issue is more related to the large number of connections, but this is a normal JavaScript project these days, it is not a huge project. Developers rely on liveload a lot these days and a slow reloading of the page really breaks the workflow. Here is a video (watch in fullhd) https://youtu.be/zbp2syKiyj0 I made a demo page http://andreicristianpetcu.github.io/foobar/app/ but it does reproduce the bug as well as the installed version :( Expected results: The page should load in under a second. Please do not mark this bug as duplicate of 1143224 :P since it is not. I tested with a Nightly that already has the fix for that bug.
Reporter | ||
Comment 1•9 years ago
|
||
If anybody is interested I plan to put a small bounty on this bug. I know I cannot cover the cost of this fix but I hope more and more people start putting bounties on features and bugs and make Firefox more independent.
Comment 2•9 years ago
|
||
I can confirm that it's slower in FF than chrome. I'm seeing something like ~2 seconds in FF (on an fx-team build with a clean profile) and ~1 second on Chrome canary in a clean profile.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: Firefox liveload slow with lots of connections → Firefox liveload with lots of connections slow when netmonitor is open
Component: Untriaged → Developer Tools: Netmonitor
Blocks: dt-open-slow
Comment 4•9 years ago
|
||
(In reply to andrei from comment #0) > User Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:38.0) Gecko/20100101 > Firefox/38.0 > Build ID: 20150605133456 > > Steps to reproduce: > ... This is an amazing bug report - thanks so much for taking the time!
Reporter | ||
Comment 5•9 years ago
|
||
I've put a modest 15$ bounty on this bug :) https://www.bountysource.com/issues/22407575-firefox-liveload-with-lots-of-connections-slow-when-netmonitor-is-open
Reporter | ||
Comment 6•9 years ago
|
||
Do you have an estimation when this bug will be fixed or if it can be fixed? If this issue gets fixed and lands in Nightly or Aurora before #1143224 lands in stable I can make a video with the performance gain of both bugs :D I think lots of JS developers switched to other browsers because of this. I know how annoyed my work colleagues are when they work on my computer and brag on how fast "their" browser is :P Thank you!
Reporter | ||
Comment 7•9 years ago
|
||
I've just put another 30$ on this bounty. It is a total of 45$ now. I guess this is peanuts for a software developer (especially if he or she is living in the US) but if more people donated small amounts as I do, maybe Firefox can become more independent. I really want this bug fixed, and I do not have the skills to fix it myself. This really breaks my flow since I loose a few seconds each time I save a file in my web project. I hope some Mozilla developers will have time for this bug soon :)
Comment 9•9 years ago
|
||
Adding DevAdvocacy keyword since there's some social proof of this being a real issue for folks on Reddit: https://www.reddit.com/r/firefox/comments/3ghmba/firefox_is_really_slow_with_liveload_i_loose_a/ Andrei: Thank you so much for the detailed report and test case. I'll take that over a bounty any day. :)
Keywords: DevAdvocacy
Comment 10•9 years ago
|
||
Hi, I was also able to replicate this. Hope it gets some attention :) Cheers, Petre
Updated•9 years ago
|
Priority: -- → P1
Comment 11•9 years ago
|
||
Brian you were the last person in the netmonitor IIRC - can you estimate difficulty for this?
Flags: needinfo?(bgrinstead)
Whiteboard: [polish-backlog]
Comment 12•9 years ago
|
||
(In reply to Jeff Griffiths (:canuckistani) from comment #11) > Brian you were the last person in the netmonitor IIRC - can you estimate > difficulty for this? Running in e10s does seem to help a bit (maybe half a second faster), which makes me think there's still some frontend jank but it isn't fixing the bulk of the problem. And there probably isn't a ton more low hanging fruit for that after Bug 1143224. Getting this to have (near)zero impact would be hard, we are always going to have some overhead for platform level monitoring and RDP traffic. But I think if we put some effort towards profiling this we should be able to get better than today with less work than that. I'm seeing something weird in the waterfall on the demo where there is a long delay before the recaptcha__en.js request happens. It looks like nearly all of the requests are done in < .5 seconds, but that one doesn't start until 2.5 seconds in. I see a similar thing on a full reload, but it's not as pronounced. I'll upload a HAR file with more info.
Depends on: 1143224
Flags: needinfo?(bgrinstead)
Whiteboard: [polish-backlog] → [polish-backlog][difficulty=medium]
Comment 13•9 years ago
|
||
HAR export - this can viewed by dropping it onto http://www.softwareishard.com/har/viewer/
Not sure if this was already the case during your last profile, but I noticed that desktop APZ (which defaults on in Nightly for Mac) makes scrolling the netmonitor much jankier and slower. Disabling APZ makes this better. Not sure if there's anything the tool itself can do for this piece of it.
(In reply to J. Ryan Stinnett [:jryans] (use ni?) from comment #14) > Not sure if this was already the case during your last profile, but I > noticed that desktop APZ (which defaults on in Nightly for Mac) makes > scrolling the netmonitor much jankier and slower. Disabling APZ makes this > better. I filed bug 1201192 about the APZ issues with net monitor.
Reporter | ||
Comment 16•9 years ago
|
||
Is there any update on this bug? I see it's a P1, does this mean it has high priority or low priority? Thank you!
Comment 17•9 years ago
|
||
I have been testing this and I can also reproduce the problem. I tried two scenarios: 1) Loading the page *with* the toolbox open (having the Net panel selected or unselected didn't make visible impact). The loading time in this scenario was ~3-4 sec, which corresponds to comment #0. 2) Loading the page *without* the toolbox open, but having HARTriggerExport extension [1] installed so, I can collect HAR timing info. The loading time in this scenarion was ~1.3 sec, which is equal to what I am seeing in Chromium. [1] http://www.softwareishard.com/blog/har-export-trigger/ These results clearly points to the Net panel UI. It looks like the Network panel logic or rather the Network panel rendering is too expensive and affects the overall page load timing - slowing it significantly down. I'd really like to work on this, but I have too much other priorities on my plate at the moment and so, I am lowering the priority to #2 - just because I don't know about anyone who could work on this now. Also adopting React in the Net panel, which is already planned, could help to solve this problem. React is fast and better equipped for rendering optimizations (comparing to XUL). --- Just a note for the scenario #2, you need to: devtools.netmonitor.har.enableAutoExportToFile = true extensions.netmonitor.har.autoConnect = true + restart the browser Honza
Updated•9 years ago
|
Priority: P1 → P2
Updated•9 years ago
|
Whiteboard: [polish-backlog][difficulty=medium] → [polish-backlog][difficulty=medium][DevRel:P2]
Reporter | ||
Comment 18•9 years ago
|
||
When is the switch to React planned? Can it be done incrementally (only some dev tools parts at first and then the rest)?
Comment 19•9 years ago
|
||
We are already switching to react (see bug 1263741 as one of the meta bugs), focusing on the Inspector panel atm. The Net panel will follow. Honza
Reporter | ||
Comment 20•9 years ago
|
||
Can you point me to the specific bugs that target the migration of the network panel to React? I want to put a bounty on their heads :D I did not manage to find them myself :P
Comment 21•9 years ago
|
||
I don't think there are reports specifically for the Net panel. So far, we have meta bug 1263750 Honza
Comment 22•8 years ago
|
||
When the Netmonitor frontend receives an event like "responseContent", it immediately fetches the response data with webConsoleClient.getResponseContent(), although this data is not needed to display the request list. It's only needed to show the request detail. So, there is a lot of useless RDP traffic. Making this fetching lazy immediately cuts the load time to half, from 4s to 2s. Chrome is still far better at 0.5s, so I hope there are more low-hanging optimizations like this.
Assignee: nobody → jsnajdr
Reporter | ||
Comment 23•8 years ago
|
||
This is really nice! Is this lazy loading in Nightly? Can I test it out? :D
Comment 24•8 years ago
|
||
(In reply to andrei from comment #23) > This is really nice! Is this lazy loading in Nightly? Can I test it out? :D Right now, it's just an unfinished patch on my machine. You'll be able to test a try build when I start testing this, and it'll get to Nightly only when it lands and gets RESOLVED FIXED.
Comment 26•8 years ago
|
||
(In reply to Jarda Snajdr [:jsnajdr] from comment #22) > Making this fetching lazy immediately cuts the load time to half, from 4s to > 2s. Chrome is still far better at 0.5s, so I hope there are more low-hanging > optimizations like this. Great find, looking forward to this improvement!
Comment 27•8 years ago
|
||
First working version of the lazy-loading patch. In my testing, it cuts the loading time to half compared to the non-patched version. Almost all mochitests are passing, except HAR and filtering ones. It will require a lot of cleanup and dividing into multiple reviewable patches, but is ready to be tested.
Comment 29•8 years ago
|
||
Andrei, when the try build in comment 28 finishes, you can download the linux64 binaries and try out how the fix works for you.
Flags: needinfo?(andrei)
Reporter | ||
Comment 30•8 years ago
|
||
So I need to download from here https://archive.mozilla.org/pub/firefox/try-builds/jsnajdr@gmail.com-34b97e789e5b1e9999c87a826f745caff080e0c9/try-linux64/ this https://archive.mozilla.org/pub/firefox/try-builds/jsnajdr@gmail.com-34b97e789e5b1e9999c87a826f745caff080e0c9/try-linux64/firefox-51.0a1.en-US.linux-x86_64.tar.bz2 right?
Comment 31•8 years ago
|
||
(In reply to andrei from comment #30) > So I need to download from here > https://archive.mozilla.org/pub/firefox/try-builds/jsnajdr@gmail.com- > 34b97e789e5b1e9999c87a826f745caff080e0c9/try-linux64/ this > https://archive.mozilla.org/pub/firefox/try-builds/jsnajdr@gmail.com- > 34b97e789e5b1e9999c87a826f745caff080e0c9/try-linux64/firefox-51.0a1.en-US. > linux-x86_64.tar.bz2 right? Yes
Reporter | ||
Updated•8 years ago
|
Flags: needinfo?(andrei)
Reporter | ||
Comment 32•8 years ago
|
||
Are you sure? I get the same performance on both Nightly and the try build. Each one has a brand new profile so there are no addons messing around with my data. The page in the project above (foobar) is ~1.7s for both builds as an average and I get 1.5 best and 2s worst but the same data is for both of them. I tried both manually loading each page with F5 so that the two builds don't mess with each others data and network and I also used the "touch app/index.html" command. Am I doing something wrong? The try build did not update itself, I did not restart it.
Reporter | ||
Comment 33•8 years ago
|
||
There has been some other things improving the performance besides your work. My initial bug report showed 3.7s on that nightly but today's nightly is 1.7s.
Reporter | ||
Comment 34•8 years ago
|
||
I run my FF instances with: ~/Downloads/firefox_liveload/firefox -P "vanilla_fast_connections" -new-window ~/firefox_nightly/firefox -P "vanilla" -new-window
Comment 35•8 years ago
|
||
(In reply to andrei from comment #32) > Are you sure? I get the same performance on both Nightly and the try build. > Each one has a brand new profile so there are no addons messing around with > my data. Hmm, do you have e10s enabled? I now realize that the patch only improves performance on e10s. It avoids unnecessary transfers of large data, and that has impact only if the transfer has significant cost. I.e., moving the data across process boundaries. On non-e10s, the transfer costs almost nothing. By the way, my latest unpatched Nightly is also significantly faster than few days ago. I'm wondering what caused that.
Comment 37•8 years ago
|
||
(In reply to andrei from comment #36) > e10s is default in Nightly so yes.... I use it. The try build has a lot of test failures that shouldn't be there - these tests succeeded locally. So there's something wrong with the pushed patch. I'll need to fix that and then try again.
Comment 38•8 years ago
|
||
(In reply to andrei from comment #34) > I run my FF instances with: > > ~/Downloads/firefox_liveload/firefox -P "vanilla_fast_connections" > -new-window > > ~/firefox_nightly/firefox -P "vanilla" -new-window You might need to do -no-remote (or close out any other running process) to make sure it's actually loading the try build and not using an existing process.
Comment 39•8 years ago
|
||
I have been also testing this using STR from comment #0 (on my own build, Win10) Unfortunately, I couldn't see any differences in page-load-speed when testing with/without the patch applied. Simple re-load of the page (pressing F5 or modifying foobar/app/index.html) takes about 2.5 s on my machine. I tried to switch on/off e10s, but didn't observe any changes... Honza
Comment 40•8 years ago
|
||
(In reply to Jan Honza Odvarko [:Honza] from comment #39) > Unfortunately, I couldn't see any differences in page-load-speed when > testing with/without the patch applied. Simple re-load of the page (pressing > F5 or modifying foobar/app/index.html) takes about 2.5 s on my machine. I > tried to switch on/off e10s, but didn't observe any changes... Very strange, I stopped seeing any improvements, too. :(
Updated•8 years ago
|
Reporter | ||
Comment 41•7 years ago
|
||
Firefox gets a bad rep due to this DevTools bug when people do benchmarks :( I seen it several times, not just in this video. https://youtu.be/ULnd_PoW-kY?t=140
Blocks: devtools-performance
Assignee: jsnajdr → nobody
Updated•6 years ago
|
Whiteboard: [polish-backlog][difficulty=medium][DevRel:P2] → [polish-backlog][difficulty=medium][DevRel:P2][qf]
Updated•6 years ago
|
Whiteboard: [polish-backlog][difficulty=medium][DevRel:P2][qf] → [polish-backlog][difficulty=medium][DevRel:P2][qf:f64][qf:p1]
Updated•6 years ago
|
Whiteboard: [polish-backlog][difficulty=medium][DevRel:P2][qf:f64][qf:p1] → [polish-backlog][difficulty=medium][DevRel:P2][qf:p1:f64]
Updated•6 years ago
|
Product: Firefox → DevTools
Updated•6 years ago
|
Whiteboard: [polish-backlog][difficulty=medium][DevRel:P2][qf:p1:f64] → [polish-backlog][difficulty=medium][DevRel:P2][qf:p2:pageload]
Updated•5 years ago
|
Reporter | ||
Comment 42•5 years ago
|
||
The touch app/index.html thing stopped working for some reason.
I ran 10 refresh by clocking the app "Reload page (old school)" on the page in the foobar repository I linked above. Here is the perf data I got.
FYI
Chrome:
Load: 217ms
Dom Content Load: 203ms
Finish: 197ms
Firefox:
Load: 1.10s
Dom Content Load: 597ms
Finish: 1.09s
My initial report was with 3.7s, my last comment was 1.7s and currently it is 1.10s. That's a pretty nice improvement but it's not yet as fast as Chrome. Hope my extra data helps.
Comment 43•5 years ago
|
||
Thanks petcuandrei for the follow-up. The profile looks much better, given most hangs are 300ms or less. We have some actionable bugs in the meta (like bug 1606514), so let's close this for now.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Updated•3 years ago
|
Performance Impact: --- → P2
Keywords: perf:pageload
Whiteboard: [polish-backlog][difficulty=medium][DevRel:P2][qf:p2:pageload] → [polish-backlog][difficulty=medium][DevRel:P2]
You need to log in
before you can comment on or make changes to this bug.
Description
•