A number of add-on developers would like to be able to import the list of search engines that have been added to Firefox's search bar.

In the Options page of our extensions, we list search engines along with the search query string url that can be customised by the user. This gives the user the most flexibility to meet his or her needs.

Not being able to import search query string urls would oblige add-on developers to implement 2 different ways of conducting searches using search engines, which is a mess and would be confusing to the user.

The addition of a queryUrl property to the existing properties (name, isDefault, alias and FavIconUrl) returned by browser.search.get() when the HTTP request method is GET, would allow add-on developers to import all search engines in Firefox using a GET method, which are the majority.

To be complete, a solution would also have to include importing search engines that use POST requests and communicating changes to the list of search engines in Firefox by firing an event of the same type as browser.storage.onChanged. This, however, should not affect adding a queryUrl property, which is a step in the right direction.

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

(In reply to Release mgmt bot [:sylvestre / :calixte / :marco for bugbug] from comment #1)

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

This bug could eventually be linked to Firefox > Search bar, but not the Address bar. Furthermore, the accurate categorization would be Webextensions > Search API.

Please can you give some example use cases for this?

I believe the intent is for WebExtensions to use search.search() rather than spinning their own. A possibility could be to pass the extra parameters to search.search()

I created an add-on called Context Search. It allows users to perform searches of selected text on a web page using a search engine from a context menu. The list of custom search engines is managed in the Options page of the extension which includes different parameters, the most important being the search query string. My extension only handles search engines that use GET requests. The search results are obtained via a fetch request. Currently, in the absence of a search query string to import from Firefox search engines, extension developers are having to process searches in two different ways depending on whether the search engine was added in the extension or imported directed from Firefox. This isn't practical at all and brings confusion to users. I've also seen some add-on developers resorting to import data from the mozlz4 file in user's Firefox profile. Besides being an ugly solution it also poses security risks because it requires file access.

Here is a small list of add-ons that would benefit from the above feature:
https://github.com/ssborbis/ContextSearch-web-ext
https://github.com/CanisLupus/swift-selection-search
https://github.com/odebroqueville/contextSearch/

I took a brief look a those extensions, so I may have missed something, but none of those seem to need access to the query parameters - they can do their searches for the built-in engines via search.search(). I also don't see any need for user required capabilities to change/interact with those.

Whilst there's a difference in the way of searching for the extension to handle, that is generally a matter of coding, but could also be addressed better by bug 1643785.

(In reply to Mark Banner (:standard8) from comment #5)

I took a brief look a those extensions, so I may have missed something, but none of those seem to need access to the query parameters - they can do their searches for the built-in engines via search.search(). I also don't see any need for user required capabilities to change/interact with those.

Whilst there's a difference in the way of searching for the extension to handle, that is generally a matter of coding, but could also be addressed better by bug 1643785.

I totally disagree with you. As I stated it's both messy to code for add-on developers, but even worse it'll be confusing for users when they visit the Options page and see that some search engines have a search query string and others don't. I guarantee you many won't understand why these differences exist.

I understand that you would like add-on developers to bare the burden of developing two methods to make searches when in all honesty it is more elegant and makes more sense to add a simple property queryUrl to the results returned by browser.search.get(). I believe this property already exists behind the scenes and that it's just a question of making it available to developers. It's not like we're asking for the construction of some pyramid!

You're welcome to disagree. I think that at the moment, it would be better to get a resolution on bug 1643785 first, and then re-assess what is desired here, hence adding that bug as a dependency.

(In reply to Olivier de Broqueville from comment #6)

I totally disagree with you. As I stated it's both messy to code for add-on developers, but even worse it'll be confusing for users when they visit the Options page and see that some search engines have a search query string and others don't. I guarantee you many won't understand why these differences exist.

It sounds like your extension already has inconsistencies and confusing behavior since, as you mentioned, it doesn't handle POST-based engines. If you use browser.search.search() everywhere and guide your users to install any additional engines using an addon or Open Search, then you have much less code to maintain users would never have to look at query strings anywhere. Additionally, users wouldn't see the inconsistency that some search engines are available through the extension but not in the location bar (or search bar). That sounds like a much better experience for users than the alternatives being offered here.

(In reply to Andrew Swan [:aswan] from comment #8)

(In reply to Olivier de Broqueville from comment #6)

I totally disagree with you. As I stated it's both messy to code for add-on developers, but even worse it'll be confusing for users when they visit the Options page and see that some search engines have a search query string and others don't. I guarantee you many won't understand why these differences exist.

It sounds like your extension already has inconsistencies and confusing behaviour since, as you mentioned, it doesn't handle POST-based engines. If you use browser.search.search() everywhere and guide your users to install any additional engines using an add-on or Open Search, then you have much less code to maintain users would never have to look at query strings anywhere. Additionally, users wouldn't see the inconsistency that some search engines are available through the extension but not in the location bar (or search bar). That sounds like a much better experience for users than the alternatives being offered here.

I first started making the extension as a service to friends on social networks who were upset that Ben Basson wasn't renewing his extension when the new Webextensions API was being released by Mozilla. I believed the open source way to do things respectful of user's privacy and adopted by Mozilla was the right way to go. I am not a seasoned developer so my extension is far from perfect, but it's my first extension and I am proud to have 9'000 happy users who don't criticise an extension they've never used while I have also sought to contribute to Firefox's success albeit in a modest way. I don't appreciate the lack of help add-on developers like myself receive here. Instead of working with add-on developers to help improve the ultimate user experience we encounter many obstacles if not a wall.

Despite having spent hundreds of hours developing my extension, I would have no problem removing it from AMO to spare users the troubles of having to cope with the inconsistencies and confusing behaviour and would be happy to explain to all the users why. Making the changes to my extension that you suggest would mean spending another very large amount of hours coding, but it would also deny users the fine grained control they currently have as they are totally in control of the search query url they want to use. It is something that I believe users want. Even the Open Search standard, which you were keen to mention, includes a query url property.

"Additionally, users wouldn't see the inconsistency that some search engines are available through the extension but not in the location bar (or search bar)."
This is why I have also filed bug 1643785. I would very much like users to have the option of still being able to use all their search engines from the search bar if they wish. Ideally, I would like to be able to sync the search engines between my extension and the search bar.

I would also like to be able to include search engines using the POST method, but I am not knowledgeable enough to develop that as, to my understanding, the data requested can vary from one search engine to another. This is also one of the main reasons why users should not give up using Firefox's search bar b/c it makes it easy to add search engines using the POST method.

Ultimately, my extension would only be temporary I guess as the features it offers which users can't find in the search bar today would eventually be baked into Firefox. I have in mind mostly multisearch and the possibility to open search results in the sidebar.

I'm not interested in criticizing your extension, apologies if that's how comment 8 sounded. I'm just trying to understand your requirements and goals. If you you interpret suggestions about alternative ways to achieve your goals as "lack of help" (I've seen people be quite generous with their time helping you on matrix) and if you respond with hostility, I think you'll find that people quickly lose interest in offering their time to help you.

Anyway, between this bug and bug 1643785, it seems that you want to offer users a different way to configure search engines. Its still not clear to me (and to other readers of these bugs I suspect) what use case you're trying to facilitate and how/why that case isn't handled well right now. I think if that was cleared up, this discussion would all become more straightforward.

You need to understand the history. When I started the extension, there was no search API available to import search engines already in Firefox and search.search() wasn't available either. To allow users to select search engines from a context menu, I had to give them a way to manage the search engines in the Options page. I figured this could only be done using a search query url. Before the search API had been implemented, there was a good proposal made by Geoffrey De Belie: https://github.com/Smile4ever/webext-experiment-searchengines. The searchUrl and Description properties were abandoned.

Was I to start a new extension today, I'd do things differently without resorting to the use of a query url. But I now have over 3'000 lines of code and many users I'm assuming who are using the extension are not using or have stopped using the Firefox search bar b/c my extension offers them more features and flexibility as to where the search results are displayed and how they can trigger searches. The only reason I am requesting a query url is because many users have complained not being able to import their search engines from Firefox automatically. Which essentially means there is a demand for the features I've implemented.

Although it is now be possible to import the search engines from Firefox without a search query url and use search.search() to carry out a search, for the search engines that are in my extension and not in Firefox, users still carry out searches using the search query url. Therefore, on the Options page, I'd end up having to list some search engines with a query url and those imported from Firefox without it.

https://i.imgur.com/pQDIe8q.jpg

I'm very concerned that this would be confusing to users. Users have a tendency not to read the documentation. (I also wonder how a browser.search.set() property can be implemented without specifying the search query url to be used. I won't discuss about POST methods b/c the very large majority of search engines use GET methods and also because I'm not proficient enough to address that case. Of the 3 extensions I mentioned previously, I think ContextSearch-web-ext has addressed this issue.)

Ideally, an extension would be able to add features and functionality without drawing users away from Firefox's built-in features such as the side bar. The best way to do that would be to provide syncing between the search engines in Firefox and those in an extension b/c it gives users the best of both worlds. Adding a search engine via the search bar remains the easiest way to perform that task. For syncing to work, there has to be an agreed upon minimal set of properties to have. IMHO, although it is not necessary for users, it can be useful for add-on developers to have access to a search query url in addition to the properties already included in the search results obtained from browser.search.get(). It also has the advantage of being in line with the Open Search standard and means that several present extensions will be able to import search engines and continue to work the way they do without resorting to importing data from the mozlz4 file in user's Firefox profiles.

https://developer.mozilla.org/en-US/docs/Web/OpenSearch

Finally, providing add-on developers access to the search query url also opens the way to new possibilities using URLSearchParams, although I haven't given that much thought.

https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams

I am going to restrict comments on this bug. Olivier, if you'd like to take this up with me directly you're welcome to email me directly, but that last line was an unwarranted escalation and not a discussion that belongs in our issue tracker.

We're unlikely to provide what was specifically asked here, since it doesn't cover all existing configurations for search.

A more broad description for search configuration could be provided, but the manner in which this bug was discussed isn't very productive toward that solution, so closing this one with the hopes that discussion in another bug might be more constructive.