User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:84.0) Gecko/20100101 Firefox/84.0

Steps to reproduce:

Every so often Firefox would change my default search engine from duckduckgo to Google. I started removing all of the search engines in preferences so that only duckduckgo remained. After some time, this list was reset again and Google was made the default again. I'm not sure if it coincides only with Firefox updates, I manually update Firefox and this happens more often than that.

Actual results:

1. search engine list resets in spite of my preference
2. default search engine set to google in spite of my preference

Expected results:

These should have stayed as I set them.

To try to work around this, I've gone into the advanced configuration and turned off everything I could find that looked like it might want to update the search engine list. If I remove things from that list they should stay removed.

It didn't help.

I ended up downloading an editor for mozlz4 (really, is this needed? you can't just leave it as JSON?) ... I removed everything there search engine related, and left only duckduckgo. Now I saved the file out and then I tried locking the file by making it read only. If I did that Firefox would delete the file.

So I used the macos command chflags uchg and that locked the file down, now it won't update anymore.

I am a programmer and I didn't install any stupid spyware that will try to change these settings.

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Also see bug 1666651

Hi, we're sorry that this is happening, it seems rare, though I've seen one or two tweets of this occurring.

How frequently does this occur normally? Could it be once or twice a month, approximately in line with updates?

Could you attach a copy of your information form Help -> Troubleshooting Information ?

Do you have any copies of search.json.mozlz4 from before it reverted?

I ended up downloading an editor for mozlz4 (really, is this needed? you can't just leave it as JSON?) ... I removed everything there search engine related, and left only duckduckgo. Now I saved the file out and then I tried locking the file by making it read only. If I did that Firefox would delete the file.

So I used the macos command chflags uchg and that locked the file down, now it won't update anymore.

I would suggest not doing this. Doing so will cause Firefox to do more work than necessary on startup. We'd rather find the cause.

Something you could try to help us narrow this down:

• Visit about:config
• Search for browser.search.log double-click to set it to true.
• Use Firefox as normal. As soon as you notice that it has switched to Google, open up the Browser Console (not web console) from Tools -> Web Developer.
• Right-click and select all, then cmd-c to copy everything
• Paste the output into an attachment here.
• A copy of the search.json.mozlz4 file before and after may be useful.

I am a programmer and I didn't install any stupid spyware that will try to change these settings.

Actually, I just had another thought. I know this is the case, but could you please use something like MalwareBytes to check for malware/adware anyway?

I'm thinking this could be a third-party that is installing an engine that is blocked, and then we're resetting the default to Google, as that is the only option we have.

I ran Malware Bytes several times at the beginning. There's no malware.

Since it was resetting to Google search every time I started by removing anything associated with Google (I had a Google Translate extension on but removed it in case it was the culprit).

It happens regularly, about once a week.

It's more than just resetting the default to Google, if I remove all the search engines except DDG from Preferences, after things are reset, all of those engines are repopulated.

I use google from time to time but only through the DDG website and use the bang searches when I don't get what I want out of DDG ... my habit is to do that back when I was using Safari so I never switched habits to use Firefox's features to switch between engines.

I'll unlock the file from changes and run for a while until it's reset given the instructions above, and report back. I'll keep copies of the search.json.mozlz4 file.

Also right now looking in my profile, a new file was added yesterday looks like, search.json.mozlz4.tmp ... and my preferences are reset again with all of the search engines showing up that I deleted. Not sure when they came back again. This time all of the deleted engines are in there but duckduckgo is still holding in place as the default engine.

It looks a bit like whatever is resetting got half the job done.

I'll post what is in the current search and the tmp mozlz4 files after this.

this is from the search.json.mozlz4.tmp that showed up.

{
    "version": 6,
    "engines": [
        {
            "_name": "Google",
            "_isAppProvided": true,
            "_metaData": {}
        },
        {
            "_name": "Wikipedia (en)",
            "_isAppProvided": true,
            "_metaData": {}
        },
        {
            "_name": "Bing",
            "_isAppProvided": true,
            "_metaData": {}
        },
        {
            "_name": "Amazon.ca",
            "_isAppProvided": true,
            "_metaData": {
                "hidden": true,
                "alias": null
            }
        },
        {
            "_name": "DuckDuckGo",
            "_isAppProvided": true,
            "_metaData": {}
        },
        {
            "_name": "eBay",
            "_isAppProvided": true,
            "_metaData": {}
        }
    ],
    "metaData": {
        "useSavedOrder": false,
        "current": "DuckDuckGo",
        "hash": "I9obs2zx5z0Dqd6/+aeWoVsoTA/PCwClpNsly4J6+dQ="
    }
}

This is my search.json.mozlz4 file... (I'm on Mac)... chmod a-w on the file as mentioned ended up with the file being deleted. I had to chflags uchg on it to stop it from being deleted or modified.

{
    "version": 6,
    "buildID": "20210105180113",
    "locale": "en-US",
    "engines": [
        {
            "_name": "DuckDuckGo",
            "_isAppProvided": true,
            "_metaData": {}
        },
        {
            "_name": "Amazon.ca",
            "_isAppProvided": true,
            "_metaData": {
                "hidden": true,
                "alias": null
            }
        }
    ],
    "metaData": {
        "useSavedOrder": false,
        "current": "DuckDuckGo",
        "hash": "I9obs2zx5z0Dqd6/+aeWoVsoTA/PCwClpNsly4J6+dQ="
    }
}

Screenshot from my search preferences... you can see DDG still holding now as default but the engines are all repopulated in the UI.

https://i.imgur.com/zWv94Uy.png

Setting the flags now as asked and will unlock the file...

I did the following steps.

1. removed the uchg flag and chmod u+w on search.json.mozlz4
2. did a file listing in my profile:

[tome] [Firefox/Profiles/i2pe7twf.default] ls -la *mozlz4*

-rwxr-xr-x@ 1 samurai  staff  373 27 Jan 12:37 search.json.mozlz4*
-rw-------@ 1 samurai  staff  279 30 Jan 20:24 search.json.mozlz4.tmp

3. quit Firefox
4. did another file listing (note search.json.mozlz4 was deleted by Firefox when it quit)

[tome] [Firefox/Profiles/i2pe7twf.default] ls -la *moz6.lz4*
-rw-------@ 1 samurai  staff  279 30 Jan 20:24 search.json.mozlz4.tmp

5. launched Firefox but did nothing else
6. file listing again, the .tmp file is gone and search.json.mozlz4 is restored, note from the file permissions it is the .tmp file being moved into place.

[tome] [Firefox/Profiles/i2pe7twf.default] ls -la *mozlz4*
-rw-------@ 1 samurai  staff  187 31 Jan 10:16 search.json.mozlz4

7. my default search engine reverted to Google when this happened

https://i.imgur.com/LTamIXt.png

8. contents of the search.json.mozlz4 are what would be expected if that .tmp file got put into place as well

{
    "version": 6,
    "engines": [
        {
            "_name": "Google",
            "_isAppProvided": true,
            "_metaData": {}
        },
        {
            "_name": "Wikipedia (en)",
            "_isAppProvided": true,
            "_metaData": {}
        },
        {
            "_name": "Bing",
            "_isAppProvided": true,
            "_metaData": {}
        },
        {
            "_name": "Amazon.ca",
            "_isAppProvided": true,
            "_metaData": {}
        },
        {
            "_name": "DuckDuckGo",
            "_isAppProvided": true,
            "_metaData": {}
        },
        {
            "_name": "eBay",
            "_isAppProvided": true,
            "_metaData": {}
        }
    ],
    "metaData": {
        "useSavedOrder": false
    }
}

I have my Firefox profiles stored on a separate encrypted drive, and I am symlinking into the encrypted drive.

[tome] [Firefox/Profiles/i2pe7twf.default] ls -la ~/Library/Application\ Support/ | grep Firefox
lrwxr-xr-x    1 samurai  staff    33 18 Aug  2019 Firefox@ -> /Volumes/untitled/Library/Firefox

My active extensions are:

1. uBlock Origin
2. Dark Reader
3. mozlz4-edit (only enabled when I need to pull content from these files)
4. Default Bookmark Folder
5. Enpass
6. Firefox Color
7. History Cleaner

... pasting the results of the log into a messages by itself. Note that during shutdown it removed the search.json.mozlz4 file that was there with the tmp file waiting around. So the first error messages are about not finding the settings file from this. I'll reset the log, put everything back to how I "want" it with DDG and no search engines and when it reverts again I'll paste the log in again.

Seeing this error coming up in the logs...

Could not write session state file  
{…}
​fileName: "(unknown module)"
​lineNumber: undefined
​operation: "open"
​path: "/Users/samurai/Library/Application Support/Firefox/Profiles/i2pe7twf.default/sessionstore-backups/recovery.jsonlz4.tmp"
​stack: ""
​unixErrno: 2
​<prototype>: Object { toString: toString(), toMsg: toMsg(), stack: "", … }
 <empty string>

Just a thought, this might have something to do with MacOS tightening permissions on what applications and programs can access in the filesystem. Firefox is still obviously able to write into the profile but seems to not be able to create folders in there. macOS does have permissions for applications to be able to access external drives as of Catalina and Big Sur increased this in general. So might be some issues with dealing with the profile once it's on a drive other than the main disk. I'll try giving it full disk access and see if that alters anything.

1. set full disk access for Firefox in preferences
2. quit and restarted Firefox
3. still getting error messages about various folders in my profile being unable to be opened
4. profile permissions:

drwx------  67 samurai  staff  2278 31 Jan 11:47 i2pe7twf.default/

5. search engine reset to Google again immediate on quit and restart, so this maybe has isolated WHEN it is happening anyway... probably I can resolve this stuff by moving the folder out of the encrypted drive but this has worked fine for 5 years with previous versions of Firefox and just started happening about 2-3 months ago if I remember right, just hit the super annoyance level at the point I submitted the bug report (plus wasn't entirely sure if it was a bug or something else).

Noted now that my profile has the quarantine flag on it.

[tome] [Profiles/i2pe7twf.default/sessionstore-backups] ls -la@
total 496
drwx------@  4 samurai  staff     136 31 Jan 15:12 ./
	com.apple.quarantine	    22 
drwx------  69 samurai  staff    2346 31 Jan 15:10 ../
-rw-r--r--   1 samurai  staff  124647 31 Jan 15:12 recovery.baklz4
-rw-------@  1 samurai  staff  124694 31 Jan 15:12 recovery.jsonlz4
	com.apple.quarantine	    22 
[tome] [Profiles/i2pe7twf.default/sessionstore-backups] 

Ongoing error messages about not being able to write a .tmp file in sessionstore-backups, and now I'm removing the quarantine flag via sudo xattr -r -d com.apple.quarantine *

Looking into my /Applications and Firefox.app is quarantined as well. Might have something to do with this.

After removing the quarantine flag it is coming right back as Firefox is writing out files. So, removed on Firefox.app as well, and restarting firefox.

I went back and I made a mistake when I was moving my profile, moved it into Library instead of Library/Application Support...

When I moved it into Application Support, now the search engines are staying as set between quit and restart. So this solves the problem.

So the combination of issues that result in incorrect behavior is:

1. Profile data is stored on removable storage
2. OS is Big Sur (possibly may happen with Catalina, I never used Catalina)
3. Firefox now has issues reading and writing information in the profile (not consistently problematic though, bookmarks are working fine and as mentioned when I locked the search.json file my changes stayed as I wanted them)

Overall seems related to Big Sur's permissions for accessing removable disks. Setting full disk access in Preferences/Privacy didn't resolve it. There is a separate setting for removable storage but can't seem to turn that on for Firefox, as it may have to request this from the OS... I'm out of my league on that and just guessing, as I don't develop apps for macOS.

For instance Dropbox shows up with requests for Removable Volumes, Network Volumes and Documents Folder. Photoshop is showing up with Documents Folder. Firefox by default had nothing and I added it via Full Disk Access.

https://i.imgur.com/UVTbcKR.png

What I have been doing is nonstandard but ideally should work as symlinks and volumes are normally transparent in UNIX style operating systems, and has been working for 5 years until Big Sur... so it's permissions related and might be worth at least noting or looking into.

I think from here the issue is isolated, and the "workaround" is to put things back to installation standard, but maybe Firefox should look into requesting disk permissions in the manner of some of these other apps. Apple is going to continue tightening the noose over time.

Thank you for all the details and analysis. As this appears to be more about the file system, I'm moving this across to the relevant component. I'm also tentatively adding it as blocking our Big Sur tracking bug, as it seems to be related there as well. This should let the right people look into this.

From what I can see there doesn't seem to be an API to request "full disk access" - but maybe I just couldn't find it.
Kershaw can you take a look. Given this is quite the corner case I don't think it should be high priority.