Split out from bug 1566600 comment #11:

I let Firefox update itself today from from 71.0b1 to from 71.0b5. Google used to be my default search engine. But after updating & restarting Firefox, the preferences page has no search engines at all. The drop-down menu for "Default Search Engine" in empty, and the "Restore Default Search Engines" button is disabled, so I don't even have a way to reset.

Debug info:

• Firefox version 71.0b1 (developer edition)
• Mac 10.11.5 (15F34)
• browser.search.region = "US"
• intl.accept_languages = "en-US, en"
• distribution.id = "Boolean" radio button selected
• cannot click "restore default" button because it's disabled(!)

arthaey, could you try doing this first to give us some debug?

• Go back into about:config
• Search for browser.search.log, click on the toggle to set it to true.
• Restart the browser.
• Go to Tools -> Web Developer -> Browser Console (not the web console).
• Right-click and select Select All
• Right-click again, and select Copy message
• Paste that into a file and attach it to this bug.

Once you've done that, lets try and fix it:

• Please go to Help -> Troubleshooting Information.
• In the Application Basics section, next to Profile Folder select the Show in Finder button.
• Double click the selected folder.
• Shut down Firefox
• Find and rename (or move to a different location) the file search.json.mozlz4 (please keep a copy, I'd like you to attach it as well).
• Then start up Firefox again.

Hopefully that should get you working. If it doesn't a second copy of the log might be useful.

Please attach the search.json.mozlz4 file - that only contains details about which search engines you have/had installed, and what gets displayed.

Assuming that all works, you might want to set browser.search.log back to false. There shouldn't be a big performance impact with it, but it is probably better in the long term to leave it at the default.

This appears similar to my issue that @Standard8 has responded to on Twitter. I have the English UK version of Firefox installed and after updates, it resets the search engine list, in particular adds back entries I explicitly removed. After the Firefox 70 update, it restore UK Chamber of Commerce and Amazon UK.

I have before and after copies of search.json.mozlz4 that I've supplied.

Thanks Anthony, I've just being trying to reproduce this and I haven't been able to yet.

A couple of thoughts spring to mind:

• Can you go into about:config, search for browser.search.region and let me know the value that is set there. It looks like it might be "CA" or "GB" from what I can work out, but it would be useful to confirm.
• If you think it is likely to happen again, can you set browser.search.log to true, and then if it does happen, go to Tools -> Web Developer -> Browser Console (not web console), and capture the full set of messages. You'll probably need to notice and do this fairly soon after the startup where it happens, otherwise we might not have the right messages there.

search for browser.search.region and let me know the value

browser.search.region = CA

If you think it is likely to happen again...

Well its happened at least 3 times since I tweeted about this earlier in the year. I've set browser.search.log = true and will keep an eye out.

(In reply to Mark Banner (:standard8) from comment #1)

arthaey, could you try doing this first to give us some debug?

By the time I got back to this, restarting Firefox also meant it updated itself from 71.0b1 to 71.0b8. My search engine list & default are normal and working again!

Sorry I don't have a repro for you anymore, but also I don't have a problem anymore. ¯\_(ツ)_/¯

It appears to have happened again today (Firefox 71.0), but didn't notice in time to check the browser log.

Anthony, could you go to Help -> Troubleshooting information, and copy the Internationalisation & Localisation section here? Also what OS are you on?

I'm starting to think this could be a some strange race condition of how we configure search currently - we're working on a new one that is a bit simpler and I've a feeling it might help this, but it'd still be nice if we can work it out.

Windows 10 Home 1903

Internationalisation & Localisation
Application Settings
Requested Locales 	["en-GB","en-US"]
Available Locales 	["en-US","en-GB"]
App Locales 	["en-GB","en-US"]
Regional Preferences 	["en-CA"]
Default Locale 	"en-US"
Operating System
System Locales 	["en-US"]
Regional Preferences 	["en-CA"]

It looks like I have a similar problem to Anthony Howe. Is there any more information I should collect/give to help?

Application Basics
Name 	Firefox
Version 	73.0.1
Build ID 	20200221145624
User Agent 	Mozilla/5.0 (X11; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0
OS 	Linux 5.4.17

browser.search.region	GB

Internationalisation & Localisation
Application Settings
Requested Locales 	["en-GB","und"]
Available Locales 	["en-US","en-GB"]
App Locales 	["en-GB","en-US","und"]
Regional Preferences 	["en-GB"]
Default Locale 	"und"
Operating System
System Locales 	["en-GB"]
Regional Preferences 	["en-GB"]

Hi jammyatjammy, please can you try the steps below to fix it, and attach the broken search.json.mozlz4 here please?

• Go to Help -> Troubleshooting Information.
• In the Application Basics section, next to Profile Folder select the Show in Finder button.
• Double click the selected folder.
• Shut down Firefox
• Find and rename (or move to a different location) the file search.json.mozlz4 (please keep a copy, I'd like you to attach it as well).
• Then start up Firefox again.

This time when I restarted, my search engines were not affected. I'll keep on copying this file when I restart until something goes wrong.

Anthony, & jammyatjammy, is it likely that you've started Firefox and then closed it really quickly? Does Firefox normally take a long time to start up for you?

I don't think so, but maybe I'll watch out for this.

is it likely that you've started Firefox and then closed it really quickly?

Not that I recall. When I start Firefox, its typically cause I will need it for the tasks at hand, so I let to go through the whole start sequence. Its very very rare I start Firefox in a mad rush to get/post something last minute before rushing out the door.

Does Firefox normally take a long time to start up for you?

What is a long time? I just closed and restarted Firefox and it took ~15s to start, display current tab, and stop spinning on some pinned tabs. I have about 11 add-ons, 3 themes, ~15 tabs open (3 pinned - twitter, slashdot, phabricator) ; the pinned tabs tend to "spin" for a little bit while I assuming they update content; I assume there is some checking for updates of add-ons and themes.

The most recent update 73 (?) the other day/week restored the unwanted search engines (Amazon UK, Chamber of Commerce, etc).

While I prefer the British English UI, I'm considering switching to the US version in order to workaround this bug and get DNS-over-HTTP by default. Also curious if this is common issue across language versions.

This time, all of my search settings are left in place, but I have the new search engine “Chambers (UK)”. This is one of the ones I end up with when the search engines are reset.

So I noticed today that the search engines I've previously removed, Amazon UK, Chambers UK, have returned. I have not yet re-deleted them in case you want to request before and after files.

(In reply to Anthony Howe from comment #18)

So I noticed today that the search engines I've previously removed, Amazon UK, Chambers UK, have returned. I have not yet re-deleted them in case you want to request before and after files.

Let's try it. I doubt they'll tell us much, but you never know. I guess you didn't capture the logs after startup?

I have recently landed one potential improvement (which will take effect from FF 75), but I'm not sure that's all of it. There's some other things in the pipeline that might help as well.

Given I don't know when the update exactly happened and when I started noticing the unwanted search engines, its hard to get a log.

Which before/after files do you want? search.json again or something else?

(In reply to Anthony Howe from comment #20)

Given I don't know when the update exactly happened and when I started noticing the unwanted search engines, its hard to get a log.

No problem.

Which before/after files do you want? search.json again or something else?

Lets just go with the search.json.mozlz4 from before.

Before re-deleting unwanted engines.

After deleting unwanted search engines.

BTW here is Browser Log from the restart after deletion, which is probably too late.

[Exception... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIChannel.open]"  nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)"  location: "JS frame :: resource://gre/modules/L10nRegistry.jsm :: L10nRegistry.loadSync :: line 772"  data: no] L10nRegistry.jsm:772:12
[Exception... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIChannel.open]"  nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)"  location: "JS frame :: resource://gre/modules/L10nRegistry.jsm :: L10nRegistry.loadSync :: line 772"  data: no] 10 L10nRegistry.jsm:772:12
SearchService.init: false
_init start
SearchService.init: false
addEnginesFromExtension: bing@search.mozilla.org
addEnginesFromExtension: Ignoring builtIn engine.
addEnginesFromExtension: chambers-en-GB@search.mozilla.org
addEnginesFromExtension: Ignoring builtIn engine.
addEnginesFromExtension: ddg@search.mozilla.org
addEnginesFromExtension: Ignoring builtIn engine.
addEnginesFromExtension: amazon@search.mozilla.org
addEnginesFromExtension: Ignoring builtIn engine.
addEnginesFromExtension: ebay@search.mozilla.org
addEnginesFromExtension: Ignoring builtIn engine.
addEnginesFromExtension: google@search.mozilla.org
addEnginesFromExtension: Ignoring builtIn engine.
addEnginesFromExtension: twitter@search.mozilla.org
addEnginesFromExtension: Ignoring builtIn engine.
addEnginesFromExtension: wikipedia@search.mozilla.org
addEnginesFromExtension: Ignoring builtIn engine.
SearchService.init: false
Unknown category for SetEventRecordingEnabled: fxmonitor
Content Security Policy: Ignoring "'unsafe-inline'" within script-src or style-src: nonce-source or hash-source specified
Content Security Policy: This site (https://bugzilla.mozilla.org) has a Report-Only policy without a report URI. CSP will not block and cannot report violations of this policy.
Content Security Policy: Ignoring "'unsafe-inline'" within script-src or style-src: nonce-source or hash-source specified
Content Security Policy: This site (https://bugzilla.mozilla.org) has a Report-Only policy without a report URI. CSP will not block and cannot report violations of this policy.
_loadEngines: start
_findEngines: looking for engines in list.json
Content Security Policy: Ignoring "'unsafe-inline'" within script-src or style-src: nonce-source or hash-source specified 2
_parseListJSON: ignoring visibleDefaultEngines value because amazondotcom is not in the jar engines we have found
_loadEngines: loading from cache directories
_loadEnginesFromCache: Loading 11 engines from cache
_addEngineToStore: Adding engine: "Google"
_addEngineToStore: Adding engine: "Bing"
_addEngineToStore: Adding engine: "Amazon.co.uk"
_addEngineToStore: Adding engine: "Chambers (UK)"
_addEngineToStore: Adding engine: "DuckDuckGo"
_addEngineToStore: Adding engine: "eBay"
_addEngineToStore: Adding engine: "Twitter"
_addEngineToStore: Adding engine: "Wikipedia (en)"
_addEngineToStore: Adding engine: "Google Encrypted"
_addEngineToStore: Adding engine: "Elite Dangerous Wiki (en)"
_addEngineToStore: Adding engine: "IMDb"
_loadEngines: done using existing cache
_init: engines loaded, writing cache
_buildCache: Writing to cache file.
SearchService.init: false
_init: Completed _init
SearchService.init: false
_buildSortedEngineList: building list
_buildSortedEngineList: using db for order
_saveSortedEngineList: starting
_saveSortedEngineList: done
getDefaultEngines: engineOrder: ({Google:1, Bing:2})
_handleIgnoreListUpdated
_buildCache: cache file written to disk.
No matching message handler for the given recipient. MessageChannel.jsm:964
Content Security Policy: Ignoring "'unsafe-inline'" within script-src or style-src: nonce-source or hash-source specified
Content Security Policy: This site (https://bugzilla.mozilla.org) has a Report-Only policy without a report URI. CSP will not block and cannot report violations of this policy.
Content Security Policy: Ignoring "'unsafe-inline'" within script-src or style-src: nonce-source or hash-source specified 3
Key event not available on some keyboard layouts: key=“i” modifiers=“accel,alt,shift” id=“key_browserToolbox” browser.xhtml
Content Security Policy: Ignoring "'unsafe-inline'" within script-src or style-src: nonce-source or hash-source specified
SearchService.init: false 2
getDefaultEngines: engineOrder: ({Google:1, Bing:2})
SearchService.init: false 2
getEngines: getting all engines
getResultDomain: responseType: "text/html" 6
SearchService.init: true
getVisibleEngines: getting all visible engines
SearchService.init: false
getSubmission: In data: "about:"; Purpose: "keyword"
getSubmission: Out data: "about%3A"
Content Security Policy: Ignoring "'unsafe-inline'" within script-src or style-src: nonce-source or hash-source specified
getSubmission: In data: "dummy"; Purpose: "undefined"
getSubmission: Out data: "dummy"
getSubmission: In data: "dummy"; Purpose: "undefined"
getSubmission: Out data: "dummy"

(In reply to Anthony Howe from comment #16)

While I prefer the British English UI, I'm considering switching to the US version in order to workaround this bug and get DNS-over-HTTP by default. Also curious if this is common issue across language versions.

I've just checked a couple of cases, and it is across multiple languages unfortunately, though that is another data point that's useful to know.

I think you'll only get DNS-over-HTTP by default if you're seen to be in the US region. You can turn it on in preferences under network settings (though I suspect you know this already).

(In reply to jammyatjammy from comment #17)

Created attachment 9131135 [details]
search.json.mozlz4 (before restart, Chambers (UK) added)

This time, all of my search settings are left in place, but I have the new search engine “Chambers (UK)”. This is one of the ones I end up with when the search engines are reset.

This is an interesting case. Your original attachment where engines are unaffected after restart shows that you have Chambers (UK) in the list, but it is hidden - which is what we'd expect to be right.

In this attachment, Chambers (UK) is completely missing - it is not mentioned in the cache anywhere. So on next restart Firefox will detect the fact there's a missing built-in engine, and rebuild the cache. As the entry for Chambers is missing, it no longer knows it is hidden, and will display it by default.

The question therefore becomes, how does Chambers get completely stripped from the cache, whilst the others appear to remain intact?

I see that you installed Wiktionary in-between the two caches, but I wouldn't have expected that to cause it - and in Anthony's case, I'm guessing extension changing isn't happening.

I'm assuming neither of you are changing the search cache manually, nor have other applications that might change the cache?

I'll have another dig in the code and a think about what might be causing this.

and in Anthony's case, I'm guessing extension changing isn't happening.

My active extensions tend remain constant, though Firefox I assume does a daily update check. They are: Dark Reader, uBlock Origin, HTTPS Everywhere, Ignore X-Frame Options, Image Search For Google, JSON Lite, Markdown Viewer Webext, RESTclient.

I have Malwarebytes Premium installed, but I seriously doubt it would modify/quarantine the search cache. Also have CCleaner free installed, which does clean assorted Firefox caches, but auto-start disabled and used only on manual demand; last time was back last fall I think.

Does Firefox support paid for placement? Are these two search engines paying to be present overriding user preferences? I can understand paying to be on a fresh install, but once a user makes a choice to remove them that should be it.

Another question: Do either of you use Firefox Sync?

(In reply to Anthony Howe from comment #27)

Does Firefox support paid for placement? Are these two search engines paying to be present overriding user preferences? I can understand paying to be on a fresh install, but once a user makes a choice to remove them that should be it.

No, this isn't on purpose. This is a bug, pure and simple.

No, I have not used Firefox sync in a few years.

jammyatjammy: I just thought of another couple of questions. To summarise, here is the new one plus the other ones:

• Can you confirm that you are not changing the search cache manually, nor have other applications that might change the cache?
• Do you use Firefox Sync?
• Have you been changing the locale/language that Firefox uses? e.g. from US English to British English
  • Or maybe changing the language settings in Preferences?
• Have you changed or reset the browser.search.region value at all?

@jammyatjammy: I now think that bug 1623599 / bug 1623597 are your issue. I'm guessing you're on a distribution build (one installed/updated by your Linux distribution), and I think it is highly likely to be those bugs. I'm working on fixing them and hoping to get a fix into the next major release.

Unfortunately they probably don't fix Anthony's issue, but should help to give me a bit clearer data as to where/when this is happening.

Apologies for the lack of communication recently. I haven't had my search engines reset recently, so there's no new data on that front. As for your checklist,

• I'm pretty sure not.
• Apparently I do use Sync, but I don't think I have it set up on any other machines.
• I haven't actively changed the language, but in about:preferences, the Language selector appears empty.
• browser.search.region appears changed (bold) to GB.

The Firefox build I'm using is the default one from nixpkgs. The build process is quite involved, and lives somewhere here.

New feedback with Firefox 75 update.

• Yesterday Firefox installed and monitored. Noticed that amazon.ca was added to my search engine list. Different. Removed.
• This morning noticed that the en-GB language package was not update automatically, because version mismatch. Manually requested update. Restarted.
• Found the amazon.co.uk. and Chambers (UK) installed.
• Finally caught the browser console log immediately following the restart.

Additional errors appeared while I was writing up my notes about Firefox 75.

Aha, langpack. I just had someone else report the same and it looks like they are using one as well

We obviously shouldn't be swapping engines in this case even though we are. One workaround would be to use the full language build.

I'll test this out a bit later and see if I can figure out a better way to handle this.

Odd. I thought I had the full UK build installed; I know I downloaded the full package from the website and installed over top of a US build. Anyway, I'm glad that this news has revealed a suspect.

(In reply to Anthony Howe from comment #37)

Odd. I thought I had the full UK build installed; I know I downloaded the full package from the website and installed over top of a US build. Anyway, I'm glad that this news has revealed a suspect.

So just to clarify, the fact you said "noticed that the en-GB language package" implies it probably isn't a full en-GB build, from your support page:

Requested Locales 	["en-GB", "und"]
Available Locales 	["en-US", "en-GB"]

This also implies to me that en-US is there in the background.

I've logged this issue with the language packs as bug 1628405, as I wanted to make it clear as to the actual issue.

If all you want is en-GB, then you should be able to install that, and you might want to remove the old language pack. It shouldn't hurt, but probably an idea to do. Also, check that Preferences -> General -> Language has the correct language set at the end.

I'm going to mark this as a duplicate of that bug, but feel free to re-open or let me know if you think I've missed something.