Open Bug 1419053 Opened 7 years ago Updated 2 years ago

[meta] Firefox's disk write behaviour is worse than other browsers' in some situations

Categories

(WebExtensions :: General, defect, P3)

Product:
WebExtensions
Component:
General
Version:
57 Branch
Type:
defect

Tracking

(Performance Impact:low)

Project Flags:
Performance Impact low

People

(Reporter: theflyingcelt, Unassigned)

References

(Depends on 1 open bug)

Details

(Keywords: meta, perf)

Attachments

(3 files)

Screenshot_4.jpg
(deleted), image/jpeg
Details
Screenshot_5.jpg
(deleted), image/jpeg
Details
tab separated log of Firefox file write activity
(deleted), text/plain
Details
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36 Steps to reproduce: installed Firefox, checked and compared to Chrome written bytes by opening the same amount of tabs and same websites. Actual results: Firefox written bytes is 2-3 times higher than Chrome. It's bad for SSD Expected results: less I/O by Firefox
Component: Untriaged → General
Product: Firefox → Core
Attached image Screenshot_4.jpg (deleted) —
Was chrome a fresh profile as well? On first install firefox won't have any http cache, site storage, etc written. If you have been using this chrome profile for a while that would have been written on your disk long ago. We either want to compare both fresh installs/profiles or both "warm" installs/profiles.
Flags: needinfo?(theflyingcelt)
No, Chrome was installed already, and so was Firefox really. I downloaded the new Firefox 57 from the website and installed it/updated it(?). I'm not sure if that can be considered an update or a fresh installation. Sorry. This morning I launched at the same time Chrome and Firefox and opened the same websites on both, about 6 tabs each. At the moment, after about 10 hours running, Chrome has been writing 800MB, and Firefox 2.25GB, therefore about 3 times more.
Flags: needinfo?(theflyingcelt)
Ok, thanks for clarifying. Disk bytes written isn't currently a performance metric we track. Maybe we should. Marking [qf] for that. This is probably session store which has been a problem in the past. We improved things a lot by at least compressing session store, but we need to be smarted about how often we write the data out. See bug 1304389.
Depends on: 1304389
Whiteboard: [qf]
Sorry for confusing you. Well yes, that's a lot of data written which is quite bad for SSDs. Great browser apart from that. I hope this will be fixed. Cheers
Session store performance is being actively worked on, so I expect this will get looked at. See bug 1330635 for the top level meta session store performance work.
Mike, do you have any more thoughts here?
Flags: needinfo?(mdeboer)
Is there actual data that the majority of Massimo's disk IO is in fact sessionstore? (I don't see that) Easy to determine how much Firefox sessionstore IO is being done - look at about:sessionstore. Disk writes to sessionstore.js over an 8 hour day: 8,768,228,640 bytes (9 GB) And you can change the session save interval via browser.sessionstore.interval - I have mine set to 1 minute = 60000
(In reply to Wayne Mery (:wsmwk) from comment #8) > Is there actual data that the majority of Massimo's disk IO is in fact > sessionstore? (I don't see that) No, but its highly likely given our session store implementation rewrites the entire store on every change (or still on a timer as you suggest???). And I don't see about:sessionstore in either FF57 or nightly 59.
Sorry guys but I don't know much about how this works here. I reported the bug but I wonder what I'm supposed to expect now. Are you developers? Will I get a solution here or a fix in next release? How does it work exactly? Thanks
(In reply to Wayne Mery (:wsmwk) from comment #8) > Easy to determine how much Firefox sessionstore IO is being done - look at > about:sessionstore. > Disk writes to sessionstore.js over an 8 hour day: 8,768,228,640 bytes (9 > GB) This is a good way to know the traffic, and I believe this can only be provided by the addon[1], which don't support fx 57 yet. I'm also looking forward the addon update :) [1] https://addons.mozilla.org/en-US/firefox/addon/about-sessionstore/
Besides, Without the about-sessionstore addon, we can still observe the traffic by seeing the byte written on the file 'recovery.jsonlz4'.
You can see that, and I can see that on my systemt. That was why I was asking the user to confirm the specific source of his file IO. If it is the same then this is merely a duplicate of bug 1304389
I'm not an expert. Bytes written by Firefox that I can see are only from task manager in Windows. I'm considering giving up Firefox for the time being as it's really eating up my SSD :(
Simple steps: 1. type in about:config and ignore the warning 2. in search field paste browser.sessionstore.interval 3. right+click that item, pick modify 4. change it to 60000 and hit OK
Done. No difference.
(In reply to Massimo from comment #16) > Done. No difference. If I remember correctly the change takes effect immediately, and changing to 60000 would cut the IO by 75% if your issue is sessionstore IO. But if there was not difference, then your problem might not be sessionstore.
Hacky but effective: Find your profile folder, sort by date modified, and see what's always floating to the top. On Mac and Linux there are loads of tools to watch exactly what an app is doing with the filesystem, but I don't know on Windows what the best tool is for that :(
(In reply to Wayne Mery (:wsmwk) from comment #17) > (In reply to Massimo from comment #16) > > Done. No difference. > > If I remember correctly the change takes effect immediately, and changing to > 60000 would cut the IO by 75% if your issue is sessionstore IO. But if > there was not difference, then your problem might not be sessionstore. This morning I turned my computer on and opened both Firefox and Chrome, same tabs (about 10) each, same websites and let them be. After only 2 hours these are the written bytes by both browsers despite having set the sessionstore value to 60000 as suggested (see screenshot on next post if I can manage to upload it here).
Sorry I don't know how to upload a screenshot with the post. Anyway, 200MB written by Chrome, and 1,4GB written by Firefox. 7 times more. Same tabs, same websites, launched at the same time 2 hours ago. :(
(In reply to Massimo from comment #14) > I'm not an expert. Bytes written by Firefox that I can see are only from > task manager in Windows. Hi Massimo, You can use the built-in tool of windows: Resource Monitor, to see the disk activity. [1] https://en.wikipedia.org/wiki/Resource_Monitor ==== Unset the bug dependency since it's not about the session restore.
No longer depends on: 1304389
Clear the ni to reduce the noise for Mike since it's not about the session restore. ==== (In reply to Massimo from comment #20) > Sorry I don't know how to upload a screenshot with the post. Massimo, you can use the 'Attach File' link on this bug page to upload :)
Flags: needinfo?(mdeboer)
Attached image Screenshot_5.jpg (deleted) —
Hi Will and thanks. I opened some tabs on Firefox and took a screenshot in resources monitor once listed by written bytes (while loading pages). Please see attached "screenshot_5.jpg" file. Thanks
Attachment #8930374 - Attachment description: Screenshot_5.jpg → Screenshot_4.jpg
(In reply to Massimo from comment #23) > Hi Will and thanks. I opened some tabs on Firefox and took a screenshot in > resources monitor once listed by written bytes (while loading pages). > Please see attached "screenshot_5.jpg" file. Looks like related to the places.sqlite, which is mainly for saving the bookmark and browsing history. Using a fresh Firefox profile and see the disk activity again can clarify this. Besides, AFAIK browser extension could also cause issues[1], but I am not sure whether the web extension API nowadays have such ability or not? Restarting the firefox with add-ons disabled can clarify this. (You can find this action in the menu button [2]) [1] Bug 489173 - Latest Google Toolbar Appears to Make places.sqlite Massive https://bugzilla.mozilla.org/show_bug.cgi?id=489173 [2] https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe-mode
(In reply to Will Wang [:WillWang] from comment #24) > (In reply to Massimo from comment #23) > > Hi Will and thanks. I opened some tabs on Firefox and took a screenshot in > > resources monitor once listed by written bytes (while loading pages). > > Please see attached "screenshot_5.jpg" file. > > Looks like related to the places.sqlite, which is mainly for saving the > bookmark and browsing history. > Using a fresh Firefox profile and see the disk activity again can clarify > this. > > Besides, > AFAIK browser extension could also cause issues[1], but I am not sure > whether the web extension API nowadays have such ability or not? > Restarting the firefox with add-ons disabled can clarify this. (You can find > this action in the menu button [2]) > > > [1] > Bug 489173 - Latest Google Toolbar Appears to Make places.sqlite Massive > https://bugzilla.mozilla.org/show_bug.cgi?id=489173 > > [2] > https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe- > mode The other day I tried to disable all add-ons and restart but disk usage was quite the same. Perhaps not 5 times Chrome but x4 yeah. I don't have google toolbar installed. I'll probably give it a try with a fresh installation of Firefox then, although I have a feeling it won't solve the issue. Anyone else having the same problem or is it just me?
Marco, is there an easy way to see why we're doing so much writing to places.sqlite?
Flags: needinfo?(mak77)
It's unclear to me if the Chrome writes count took into account all of its processes or just one. The numbers reported in comment 23, while not extremely nice, don't look critical by themselves, it's less than 300KB/s for the larger db we have. While there is likely a relation with the 4 times more writes, that relation is not well defined. We should run the measurement for an extended timeframe and split usage by percentages to prioritize work. It's surely worth investigating and I agree with Ben, we should have a Talos test for the amount of I/O (but I actually think we have something already with xperf?) (In reply to :Gijs from comment #26) > Marco, is there an easy way to see why we're doing so much writing to > places.sqlite? We can check the queries traffic through MOZ_LOG=mozStorage:5 and MOZ_LOG_FILE set to a local file path. That will store any queries executed by Firefox (may grow quite a bit). There are surely known deficiencies in the schema and the way we store things, that we are constantly working on. Also WAL gives us a little bit more speed, but may write a bit more data. If there is no specific user interaction causing large I/O (moving bookmarks, restoring...), the things that could be running are: 1. history additions 2. additional data stored for Activity Stream (that other browsers may not store at all) 3. if the db reached its size limit (about 70MB) expiration may be running 4. If Sync is enabled, it can modify both history and bookmarks 5. Add-ons can clearly modify history and bookmarks 6. frecency and input history decay every day, on idle-daily IIRC (one-shot) 7. bookmarks backups happen almost every day on idle (one-shot)
As a personal suggestion, I'd tell to not worry too much about SSD tearing in itself, the current units can easily last far more time than your computer average life (10 years and more) even at large write ratios. Regardless of that, we'll work on improving I/O, because many users are still on mechanical disks that are MUCH slower.
Joel, how is our test situation regarding I/O measurement? David, I don't know if you're the best person to ask to, I'm looking for someone who could organize a small investigation team to evaluate I/O and identify some bottlenecks to be filed in the right components
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(mak77)
Flags: needinfo?(jmaher)
Flags: needinfo?(ddurst)
Keywords: perf
Product: Core → Firefox
Is context graph or activity stream hitting places harder than we used to?
I would also note that this problem would be much worse on mobile. Those devices have less performant flash and some manufacturers don't use flash friendly file systems.
Depends on: 1420571
(In reply to Ben Kelly [:bkelly] from comment #30) > Is context graph or activity stream hitting places harder than we used to? For writes I just filed one bug related to the new AS code. Surely storing the page description for every page, rather than just bookmarks, increases our writes. We could evaluate reducing the current description limit from 1024 chars to 512? Mardak? Most of the new traffic should be reads btw.
Flags: needinfo?(edilee)
Depends on: 1420597
At least currently, activity stream is able to show up to 3 lines of description text. Looks like 256 characters "should be enough for everybody". ? nanj
Flags: needinfo?(edilee) → needinfo?(najiang)
for measuring IO we have the xperf job which runs on windows 7 only- this has been running for many years and measures main thread, non main thread, startup, and everything else. While this isn't linux or android, it should give us a good idea of how much IO we are doing- There is minimal information in the logs, but if there are specific questions, I can usually hack something up and get a try push to get more specifics without too much trouble.
Flags: needinfo?(jmaher)
(In reply to Joel Maher ( :jmaher) (UTC-5) from comment #35) > for measuring IO we have the xperf job which runs on windows 7 only- this > has been running for many years and measures main thread, non main thread, > startup, and everything else. From the description here https://wiki.mozilla.org/Buildbot/Talos/Tests#xperf it seems to only measure files used on startup, not during a common browsing session. It would be extremely valuable to measure a whole session that maybe also includes some specific copies of top N websites. Is that right?
Flags: needinfo?(jmaher)
Isn't the resource-usage.json which can be found as artifact on any test job a valuable resource to identify disk I/O usages? Or what kind of usage do we record there? Here an example: https://public-artifacts.taskcluster.net/ScW9FpExRWKVF-AjcghdEw/0/public/test_info//resource-usage.json
we actually measure both startup and normal operations. Take a recent job for example: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=cad9c9573579698c223b4b6cb53ca723cd930ad2&filter-searchStr=xperf&selectedJob=147795786 we run the tp5 pageset and collect the IO. you can see: main_startup_fileio nonmain_startup_fileio main_normal_fileio nonmain_normal_fileio we also connect netio as well :)
Flags: needinfo?(jmaher)
(In reply to Ben Kelly [:bkelly] from comment #31) > I would also note that this problem would be much worse on mobile. Those > devices have less performant flash and some manufacturers don't use flash > friendly file systems. Fortunately we don't use any of the same code on iOS, or any desktop code for bookmarks or history on Android, and neither platform does significant work when not foregrounded.
Depends on: PlacesIO
No longer depends on: 1420571, 1420597
https://bugzilla.mozilla.org/show_bug.cgi?id=814942#c8 User image Nicholas Nethercote [:njn] 2012-11-30 04:17:05 IST "I/O Read Bytes" means "The number of read input/output operations generated by the process, including file, network, and device I/Os", according to http://windows.microsoft.com/en-US/windows-vista/What-do-the-Task-Manager-memory-columns-mean. So this may not be actual disk reads.
Priority: -- → P3
Whiteboard: [qf] → [qf:f63][qf:p3]
Sorry, missed the NI previously. Clearing as my input is now moot.
Flags: needinfo?(ddurst)
Today, I notice around 2GB (yes, GB) data written into disk per half hour by Firefox 59.0.1 today. The data are not written into Firefox cache but into my profile folder (I have set the cache into ramdisk). I did not notice this behaviour before. I had about 20 tabs opened. After closing Firefox and opening it again, the writing continues. But when I bookmarked all the opened tabs and planned to close all tabs to see the difference, the writing stopped before I close all the tabs. I am not sure what happened. So I comment here for reference. If I ran into this situation again, I will report back here.
the title is a bit misleading. Some work happened, but we need a deeper analysis of all the file accesses to make this actionable
Summary: Firefox writes 3 times as much bytes on disk compared to Chrome. Bad for SSD → Firefox may write more bytes on disk compared to Chrome
Whiteboard: [qf:f63][qf:p3] → [qf:f63][qf:p3][fxperf]
Seems like we should try to investigate this for fxperf.
Whiteboard: [qf:f63][qf:p3][fxperf] → [qf:f63][qf:p3][fxperf:p2]
Is there any update on this? I have the exact same issue, been measuring I/O writes since the release of Quantum. I can confirm Firefox writes about 3times more data on my HD compared to Chrome, Opera, Vivaldi.
My session recovery jsonlz4 file is <1MB and I still have lots of disk IO - as seen with disk IO light on laptop. Nightly build. Currently no memory pressure, only using 13 of 20GB.
Severity: normal → major
I'm on a very slow HDD, and every avoidable write/read operation is noticeable.
I checked yesterday: The problem is much worse, Firefox writes x5 more than Chrome on my HDD. I tried disabling extensions but nothing happens. Interestingly, on safe mode the problem is not so big but still exists.
Hi folks, the most useful information we can have here is details about exactly what is being written, and maybe what causes it. General comments like "x amount more" doesn't really help and just adds noise. For instance, if you're seeing a lot of accesses to places.sqlite (or similar files): then it is likely one of the dependencies of bug 1421562. If you're able to identify a different file that has lots of accesses, then please file a bug and mark it as blocking this one - it can then be investigated on its own and helps with identifying the causes (different people are likely to see different bugs).
How can we see which files are accessed? I'm not that tech savvy and I doubt the problem is so difficult to replicate anyway. I've tried clean installations, etc. As soon as I add an addon, ANY addon, I/O bytes reach 1GB in under 20mins of use (Chrome barely reaches 200MB in the same duration). I had to change my profile path to my HDD instead of SSD in order to prolong the latter's lifespan (and yes, I know modern SSDs last for years but with FF writing so much, I'm being careful). So let me recap: In safe mode the problem is still present but to a much smaller extent, I/O writing is decent. If I try a clean installation, and resetting the about:config to its default state the problem nearly doesn't exist. But as soon as I add any addon (I've tried different ones) then the I/O writing frenzy begins.
(In reply to Konssam from comment #50) > How can we see which files are accessed? https://docs.microsoft.com/en-us/sysinternals/downloads/procmon should be able to help, see e.g. https://www.winhelponline.com/blog/process-monitor-track-events-generate-log-file/ for some basic introductory remarks about the tool. > I'm not that tech savvy and I doubt > the problem is so difficult to replicate anyway. I've tried clean > installations, etc. As soon as I add an addon, ANY addon, FWIW, with this type of thing at least 1 example of an add-on you tried this with is going to be helpful, in order for people to be able to reproduce this themselves (so they don't end up managing to pick some other add-on that you've not tested with that doesn't show the problem). > I/O bytes reach > 1GB in under 20mins of use (Chrome barely reaches 200MB in the same > duration). "I/O bytes" meaning what? Measured how? Is this both reads and writes or only writes?
Flags: needinfo?(konsservi)
(In reply to :Gijs (he/him) from comment #51) > (In reply to Konssam from comment #50) > > How can we see which files are accessed? > > https://docs.microsoft.com/en-us/sysinternals/downloads/procmon should be > able to help, see e.g. > https://www.winhelponline.com/blog/process-monitor-track-events-generate-log- > file/ for some basic introductory remarks about the tool. > > > I'm not that tech savvy and I doubt > > the problem is so difficult to replicate anyway. I've tried clean > > installations, etc. As soon as I add an addon, ANY addon, > > FWIW, with this type of thing at least 1 example of an add-on you tried this > with is going to be helpful, in order for people to be able to reproduce > this themselves (so they don't end up managing to pick some other add-on > that you've not tested with that doesn't show the problem). > > > I/O bytes reach > > 1GB in under 20mins of use (Chrome barely reaches 200MB in the same > > duration). > > "I/O bytes" meaning what? Measured how? Is this both reads and writes or > only writes? I am talking about I/O writes. I/O reads is normal. I've searched online and realised people have been complaining about constant I/O writes in FF from 2005. On my PC it keeps writing data non-stop. I've opened FF 15 mins ago and already it has written 1.5GB (Chrome at the same period 200MB). Extensions I'm using are Lastpass, Evernote, Ghostery (I've also tried disabling Ghostery and installing UBlock origin but still the same problem persists).
Flags: needinfo?(konsservi)
(In reply to Konssam from comment #52) > (In reply to :Gijs (he/him) from comment #51) > > (In reply to Konssam from comment #50) > > > How can we see which files are accessed? > > > > https://docs.microsoft.com/en-us/sysinternals/downloads/procmon should be > > able to help, see e.g. > > https://www.winhelponline.com/blog/process-monitor-track-events-generate-log- > > file/ for some basic introductory remarks about the tool. > > > > > I'm not that tech savvy and I doubt > > > the problem is so difficult to replicate anyway. I've tried clean > > > installations, etc. As soon as I add an addon, ANY addon, > > > > FWIW, with this type of thing at least 1 example of an add-on you tried this > > with is going to be helpful, in order for people to be able to reproduce > > this themselves (so they don't end up managing to pick some other add-on > > that you've not tested with that doesn't show the problem). > > > > > I/O bytes reach > > > 1GB in under 20mins of use (Chrome barely reaches 200MB in the same > > > duration). > > > > "I/O bytes" meaning what? Measured how? Is this both reads and writes or > > only writes? > > I am talking about I/O writes. I/O reads is normal. I've searched online and > realised people have been complaining about constant I/O writes in FF from > 2005. On my PC it keeps writing data non-stop. I've opened FF 15 mins ago > and already it has written 1.5GB (Chrome at the same period 200MB). > Extensions I'm using are Lastpass, Evernote, Ghostery (I've also tried > disabling Ghostery and installing UBlock origin but still the same problem > persists). Can you answer the other questions I asked, ie how you're measuring I/O writes, or what files you're seeing being written to? Thank you.
Flags: needinfo?(konsservi)
(In reply to :Gijs (he/him) from comment #53) > (In reply to Konssam from comment #52) > > (In reply to :Gijs (he/him) from comment #51) > > > (In reply to Konssam from comment #50) > > > > How can we see which files are accessed? > > > > > > https://docs.microsoft.com/en-us/sysinternals/downloads/procmon should be > > > able to help, see e.g. > > > https://www.winhelponline.com/blog/process-monitor-track-events-generate-log- > > > file/ for some basic introductory remarks about the tool. > > > > > > > I'm not that tech savvy and I doubt > > > > the problem is so difficult to replicate anyway. I've tried clean > > > > installations, etc. As soon as I add an addon, ANY addon, > > > > > > FWIW, with this type of thing at least 1 example of an add-on you tried this > > > with is going to be helpful, in order for people to be able to reproduce > > > this themselves (so they don't end up managing to pick some other add-on > > > that you've not tested with that doesn't show the problem). > > > > > > > I/O bytes reach > > > > 1GB in under 20mins of use (Chrome barely reaches 200MB in the same > > > > duration). > > > > > > "I/O bytes" meaning what? Measured how? Is this both reads and writes or > > > only writes? > > > > I am talking about I/O writes. I/O reads is normal. I've searched online and > > realised people have been complaining about constant I/O writes in FF from > > 2005. On my PC it keeps writing data non-stop. I've opened FF 15 mins ago > > and already it has written 1.5GB (Chrome at the same period 200MB). > > Extensions I'm using are Lastpass, Evernote, Ghostery (I've also tried > > disabling Ghostery and installing UBlock origin but still the same problem > > persists). > > Can you answer the other questions I asked, ie how you're measuring I/O > writes, or what files you're seeing being written to? Thank you. Sure. I'm using Process Monitor and Process Explorer to measure I/O. So I tried this. I opened Firefox. Visited Facebook only for a 10 seconds and closed the tab. So I waited with an empty tab open to see what's going on: Firefox keeps writing non-stop. The files written mostly are extension files ending in extensions\storage.js.tmp for different extensions. What strikes me as odd is that I have installed which I have disabled because I rarely use them and these extensions are also writing excessively I/O bytes. Again they end in storage.js.tmp. Also a lot of data is written from extensions this time ending in: kv.sqlite-wal. Finally, the rest of the files which also keep writing non-stop data are cache\entries and cache/index.tmp So I am wondering: how is it possible that cache data keeps being written with only an empty tab open? I mean, it IS an empty tab, I left it open for half an hour and all this time data was being written. And why are disabled extensions so active in writing data? Thanks for your help!
Flags: needinfo?(konsservi)
Sorry, typo: I meant I have installed extensions that I have disabled.
:ddurst, can someone from the webext team take a look given comment #54?
Flags: needinfo?(ddurst)
The columns are "file name", "total bytes written to this file", "number of write transactions" with summary lines at the end.
I would also like to add my experience to the discussion. I am seeing 15 to 20GB/day written to my Firefox profile. This is for Firefox (and also Chrome), but IE 11 shows much less, around 1 GB/day. In my opinion, this is a serious issue for Firefox. I've had two SSD modules fail about 14 months apart. At 20 GB/day, that works out to 8.4 TB over 14 months, which I suspect is near the write limit for the 256 GB SSD modules that failed. To address the SSD failure problem, I've moved my profile from the SSD to a HDD and that works fine with no loss in performance that I can detect. Unfortunately, I don't have that option on two other systems which are limited to one (now SSD) drive. To understand the problem better, I used Process Monitor to record all the writedisk events for Firefox over several hours of my normal use, which includes continuous Gmail, some searching for parts on Amazon and eBay, and news reading. I used an Awk program to parse the CSV result and create a tab separated table. The most heavily written to file is ..\storage.js.tmp at over 3.4GB in 9 hours. The total bytes written was 9.18 GB, which works out to about 23 GB/ 24hr day. I've attached the txt table. The columns are "file name", "total bytes written to this file", "number of write transactions" with summary lines at the end. This will at least show the magnitude of the issue.
It looks like this heaviest file is the storage of Adblock+.
ssd failing in a so short time is unlikely to be directly related to this, there are companies testing these with continuous writes over the whole disk, and they last multiple years even in that worst case scenario. SSDs can move data around to avoid writing continuously to the same cells. Comment 59 already pointed out that in your case it looks like the largest writes are from AB+, you may start evaluating replacing it. The second large writer in your file is cookies. You could maybe disable third party cookies. The third is Places, that is already tracked. Unfortunately the various cache2 writes are quite sparse, but summing them up is likely to give a large number. This is the browsing cache used by network. We'll continue investigating and improving the situation as far as possible.
And, looking closely at what changes in ..\storage.js.tmp, it looks like the count of blocked ads is the only change and that is one or two bytes out of the whole file. The ..\storage.js.tmp file is about 4MB in size. Every time it is updated, it is entirely re-written. And, it looks like at every update it is being written twice about 0.02 seconds apart. The interval between updates is variable, but works out to an average of 8MB every 20 seconds. And this is just for the 1 or 2 bytes that change between updates. Clearly it is not Firefox's fault that Adblock+ is behaving this way. However, is there anything Firefox can do to mitigate potential SSD problems caused by this kind of behavior? Is there a general way to reduce unnecessary writes that would also help reduce the physical cashe2 writes? It's just something to think about. And, I do appreciate that you are working to improve the situation as far as possible.
Depends on: 1510687
(In reply to walterhw50 from comment #61) > And, looking closely at what changes in ..\storage.js.tmp, it looks like the > count of blocked ads is the only change and that is one or two bytes out of > the whole file. > > The ..\storage.js.tmp file is about 4MB in size. Every time it is updated, > it is entirely re-written. And, it looks like at every update it is being > written twice about 0.02 seconds apart. The interval between updates is > variable, but works out to an average of 8MB every 20 seconds. And this is > just for the 1 or 2 bytes that change between updates. > > Clearly it is not Firefox's fault that Adblock+ is behaving this way. > > However, is there anything Firefox can do to mitigate potential SSD problems > caused by this kind of behavior? Is there a general way to reduce > unnecessary writes that would also help reduce the physical cashe2 writes? > It's just something to think about. I filed bug 1510687 to consider throttling the write frequency for the storage files for add-ons. (the current throttle is 1.5s, which is just what happens to be the default for that particular storage API, which is fine for Firefox's own use, but apparently not for add-ons...)
Whiteboard: [qf:f63][qf:p3][fxperf:p2] → [qf:f63][qf:p3][fxperf:p3]
Flags: needinfo?(ddurst)

Hi, is there any update on this? I've tried measuring I/O again using the latest version of Firefox beta (71.0b3) and still FF writes much more than other browsers.

(In reply to Konssam from comment #63)

Hi, is there any update on this? I've tried measuring I/O again using the latest version of Firefox beta (71.0b3) and still FF writes much more than other browsers.

Add-on data writing was moved to indexeddb in bug 1488825 , which shipped in 66, and should have helped here, I think. There were probably other efforts since the last significant update here, but in comment #54 you indicated that this is where you were seeing write activity.

Where are you seeing this write activity now?

Because unfortunately, just reporting that we write more than "other browsers" is not really enough to dig deeper, I'm afraid. How much data we write likely depends significantly on which websites you're loading, and in some cases on local preferences/settings in your profile. For instance, I've seen bug 1583384 result in excessive memory use, and could see a similar problem creating excessive disk writes, but we'd need more details to be sure and to figure out how best to mitigate. Even in this bug we previously thought that the issue was related to session restore, when later comments indicated it was in fact related to extensions writing to disk.

Flags: needinfo?(konsservi)

I ran a set of experiments in late 2017 comparing Firefox and Chrome while running Speedometer 2, and one of the measurements was filesystem writes.

I found basically that Firefox and Chrome wrote basically the same amount of data over the course of the test. However, Chrome wrote that data in bigger chunks far less often. Firefox wrote the same amount of data in small amounts constantly.

Not sure if that's a useful or useless datapoint... maybe it just underscores Gij's comment that pathological behavior is often very much related to specific site behavior.

That said, knowing which subsystems such as cookies or addons, etc allow constant writes instead of periodic batched writes seems like it would be worth knowing.

It's been nearly a year so I assume we won't hear back. If people re-find this bug and have specific issues, please file new bugs and mark them as dependency / see also.

Flags: needinfo?(konsservi)
Keywords: meta
Summary: Firefox may write more bytes on disk compared to Chrome → [meta] Firefox's disk write behaviour is worse than other browsers' in some situations
Performance Impact: --- → P3
Whiteboard: [qf:f63][qf:p3][fxperf:p3] → [fxperf:p3]

In the process of migrating remaining bugs to the new severity system, the severity for this bug cannot be automatically determined. Please retriage this bug using the new severity system.

Severity: major → --
Severity: -- → N/A
Product: Firefox → WebExtensions
Whiteboard: [fxperf:p3]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: