STR:

1. Install an older version of a WebExtension with a search engine in it, e.g. I installed startpage 1.2.0 from here.
2. Go to Preferences, Search, and then set it as default, change the order where it is, uncheck the tick box (hidden status) and add an alias.
3. Restart so that settings are definitely saved (possibly don't need to do this step).
4. Close the preferences tab.
5. Go to about:addons and from the cog icon select "Check for Updates"
6. Go back to the preferences tab.

Expected Results

All settings are saved.

Actual Results

The hidden status and alias have been lost. The engine does remain as default and in the same position in the list of engines.

The main issue here is that when a WebExtension is updated, the Add-on manager is first calling removeEngine, and then calling addEnginesFromExtension:

https://searchfox.org/mozilla-central/rev/7d0c94a0e9a9fe1f83553f49b10128567d21709d/browser/components/extensions/parent/ext-chrome-settings-overrides.js#481,485

That code also explains why the position and default engines are kept.

We should probably add tests for:

• Updating from one version of a WebExtension to another, check that: all user settings retained, all urls/names etc updated.
• Updating from one version without a search engine to one with a search engine, check that: new engine is correctly added.
• Updating where the search engine is removed in the new WebExtension, check that: old engine is correctly removed.

I don't think the latter two will be happening often, but I think we should check for it anyway.

https://hg.mozilla.org/mozilla-central/rev/63b9f90d47b7