We need a better way to store prefs. Currently, the full prefs file gets written & fsynced on the main thread during startup, shutdown, and during explicit flushes. The new prefs storage system should periodically write prefs changes to disk incrementally & off the main thread.

Why are we writing/syncing during startup? That seems like a complete waste. In general I think we should be writing this file from a helper thread when a pref changes, instead of waiting for shutdown. Then normally I would hope that prefs are clean at shutdown and don't need to be rewritten, although we should measure that. I'm hoping we can solve this without changing the pref storage format, since that will affect a bunch of secondary tooling.

Unless I am mistaken, we are currently somewhat paranoid about writing+flushing preferences all the time because we want to ensure that preferences are never corrupted. I feel that simple snapshotting (off the main thread, of course) could help us ensure that we always have a safe set of preferences to fallback to in case of data corruption.

I'm not sure what "all the time" means. We write the pref file (on the main thread) when somebody explicitly calls .savePrefFile (which is a bit unusual) and at shutdown. I think we're not nearly paranoid enough about it ;-)

I misphrased comment 2. I meant that: - we currently perform writes on the main thread for safety reasons; - we even flush on the main thread for safety reasons. ... and I believe that just having a snapshot handy would be quite sufficient and could be done entirely off the main thread.

I think async pref writes would be a good start here. We should look into reducing fsyncs next.

> ... and I believe that just having a snapshot handy would be quite > sufficient and could be done entirely off the main thread. huh? why should we have any copies other than the hash table and the file? you have to at least enumerate the hashtable on the main thread. One would really hope pref changes are the exception, so a push based approach would seem a lot better than polling.

I've been thinking about this for quite some time: The prefs are missused(?) as a state (and also simple database, mostly in addons and devtools) storage in some cases, e. g. (just a quick look at my current about:config) all those «app.update.lastUpdateTime.*», «idle.lastDailyNotification», «places.database.lastMaintenance* or «data.reporting.healthreport.lastDataSubmission*» (and maybe «browser.slowStartup.*», «network.dnsCache*», «dom.mozApps.used» and «browser.*.*Version» and the like), also «devtools.responsiveUI.*», «pdfjs.database» and maybe «print*» etc. pp. IMHO these can be moved to a separate storage cleaning up the prefs list (and thereby making it more like a pref list than a dumpster) which is also optimized (at least for the state case) to hold volatile data which also does not need to be preserved 100% of the time (i. e. corruption won't loose too relevant information). So a part of this project can be (at least) creating a storage system for volatile state information. This may remove some pressure from the pref system so there's less to worry about the perf issues here (though the pref system still needs improvements). I don't know if it will actually speed up the system but it should not slow it down, of course.

> In general I think we should be writing this file from a helper thread when > a pref changes, instead of waiting for shutdown. This is what happens now. There are still some improvements that could be made, but I think a "new storage system for prefs" is more than we need. So I will WONTFIX this bug.