User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.87 Safari/537.36

Steps to reproduce:

enter a string in a commonly viewed URL in the awesome bar

example url: youtube.com/feed/subscriptions
example string: you

Actual results:

the first, highlighted result is 'youtube.com'

(see attachment)

I have to use use the down arrow to select the desired site.

Expected results:

I never want to visit the youtube.com homepage, I always want to visit the url youtube.com/feed/subscriptions

However the 'visit' result is always the domain name, regardless of how infrequently I visit it, vs sub URLs.

In Chrome, for example, the first result will be youtube.com/feed/subscriptions, based on my browsing history.

This will be the case for many websites, where the full url is useful, but the domain name is not.

If the decision has been made to not use the search of previously browsed sites as the primary search option (as is the case in Chrome), then I would suggest at least search and suggest bookmarks before suggesting generic domain name. In the example above, I have youtube.com/feed/subscriptions bookmarked, but the first result is still youtube.com

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

So, this is how the system is supposed to work, we autofill "to the next slash", thus you could potentially do "y(right)f(right)s(right)" to go to your url. I admit for people coming from other browsers doing url autofill it's not the best experience.
Autofilling urls would be a lot more expensive for us, and in some cases it would do a worse job than autofilling origins (it really depends on the site).
An idea I suggested in the past was to use additional data from Adaptive History to autofill urls, that would at least be a stricter set of urls interesting to the user.
I actually thought we had already a bug on file, but I can't find it (feel free to dupe if someone finds it) so for now I'll confirm this as an investigation target, because it's a quite common request.

(In reply to Asif Youssuff from comment #3)

Came up again here: https://www.reddit.com/r/firefox/comments/eqwyub/permanently_delete_suggestion/

FWIW, that was also me posting there

This is still reproducible in Firefox 75. Sigh.

Copying my comment from my duped bug, I think one reasonable compromise here would be to autocomplete bookmarked URLs.

Or, an opt in options parameter to prioritise bookmark URLs (and/or recent history URLs) above origins?

(In reply to Marco Bonardo [:mak] from comment #2)

So, this is how the system is supposed to work, we autofill "to the next slash", thus you could potentially do "y(right)f(right)s(right)" to go to your url. I admit for people coming from other browsers doing url autofill it's not the best experience.

Having to move from letters to arrows THREE times is a terrible experience regardless of where you come from. It's literally better to move the mouse at that point.

The main factor should be the frequency of visit.. no matter what the domain or path of the url is.. which is what Chrome implements.
If the user visits the homepage too often than it would be the default when autofilling, if they visits some other path or page more frequently then that should be the default.
Example: My ISP's login page is '10.254.254.4/0/up' so I open that frequently. But, I have never opened the homepage '10.254.254.4'.
When I typed '10' and pressed enter without checking, it tried to open '10.254.254.4' but showed Firefox's Security Warning, which may seem scary to unsuspecting users.

First time here, but there's a temporary workaround to this issue. See: https://github.com/xiaoxiaoflood/firefox-scripts#userchromejs-scripts and the Enter Selects script.

(In reply to Marco Bonardo [:mak] from comment #2)

So, this is how the system is supposed to work, we autofill "to the next slash", thus you could potentially do "y(right)f(right)s(right)" to go to your url. I admit for people coming from other browsers doing url autofill it's not the best experience.
Autofilling urls would be a lot more expensive for us, and in some cases it would do a worse job than autofilling origins (it really depends on the site).
An idea I suggested in the past was to use additional data from Adaptive History to autofill urls, that would at least be a stricter set of urls interesting to the user.
I actually thought we had already a bug on file, but I can't find it (feel free to dupe if someone finds it) so for now I'll confirm this as an investigation target, because it's a quite common request.

So, this is how the system is supposed to work,

If I turn off literally every type of search suggestion except "bookmarks" I get search suggestions to go to sites I haven't bookmarked - and they're not in my browser history, I cleaned up very thoroughly before testing this.

E.g. I have bookmarked "https://discord.com/channels/@me"; typing "dis" it suggests "discord.com" - a site I never wish to visit, before it suggests the actual bookmark.)

This may be how it's "supposed to work" but it's dog-awful.

I don't understand how it could possibly be "more expensive" to only show bookmarks than it is to do string analysis on the bookmarks to see what the top level domain is and present that.

To demonstrate:

four search suggestions

The problem is not that the discord bookmark isn't on top. That two non-discord bookmarks are presented first annoy me but I recognize that you'd have to read my mind to put the discord bookmark up top.

The problem is that it looks, at a glance, like hitting enter will take me to the site I want to go to, which is absolutely not what will happen.

I keep talking about Discord but it's true in general. I keep going to "reddit.com" because I have "https://www.reddit.com/r/cpp_questions/new/?f=flair_name%3A%22OPEN%22" bookmarked. I never wish to visit "reddit.com."

In my opinion it should only autofill the entire URL when the user has bookmarked itand has disabled all suggestions except for bookmarks. Otherwise, I always want to go to the base domain, not that one YouTube video I watched two months ago.
So in the default configuration, I believe it suits me and others best.

There currently exists a related configuration:

Search Suggestions

"Search Suggestions" already contains a configuration for where the suggestions are located ("Show search suggestions ahead of browsing history in address bar results").

Would it be possible to have a second, opt in, configuration here to "Show search suggestions at top of address bar results"

Under this setting suggestions, including bookmarks, would show before the autofill origin?

This would be considered a reasonable workaround for people who expect certain frequently visited URLs, either from history, or bookmarks. Users could deselect certain suggestions from appearing under "Change settings for other address bar suggestions" to customize this behaviour.

Depends on D144393

Depends on D144393

Depends on D145702

Hello Megan. Could you review this? Thanks!

Comment on attachment 9275665 [details]
data-review-request-for-bug1597791.md

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?
   Yes, this will be documented in the probe dictionary.

2. Is there a control mechanism that allows the user to turn the data collection on and off?
   Yes, users can opt out of telemetry collection.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?
   Permanent data collection, monitored by the Firefox Suggest User value team.

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
   Category 2, interaction data.

5. Is the data collection request for default-on or default-off?
   Default on.

6. Does the instrumentation include the addition of any new identifiers?
   No new identifiers.

7. Is the data collection covered by the existing Firefox privacy notice?
   Yes.

8. Does the data collection use a third-party collection tool?
   No.

data-review +

https://hg.mozilla.org/mozilla-central/rev/92cdeadc813e
https://hg.mozilla.org/mozilla-central/rev/d89c6935448b
https://hg.mozilla.org/mozilla-central/rev/d6c3afccb743

Some constructive feedback on this one: please consider making this optional because, for example, I'm used to just typing "i", which makes "instagram.com/" autofill, then I simply hit the right arrow key on my keyboard and from there I can very quickly and efficiently get to any instagram account I want. But now when I hit "i", "instagram.com/explore/tags/formula1/" is autofilling... For me, this is a huge downgrade in speed and UX efficiency. And I'm having the exact same issue with twitter.

(In reply to Will from comment #30)

Some constructive feedback on this one: please consider making this optional because, for example, I'm used to just typing "i", which makes "instagram.com/" autofill, then I simply hit the right arrow key on my keyboard and from there I can very quickly and efficiently get to any instagram account I want. But now when I hit "i", "instagram.com/explore/tags/formula1/" is autofilling... For me, this is a huge downgrade in speed and UX efficiency. And I'm having the exact same issue with twitter.

Fortunately there's one: it's browser.urlbar.autoFill.adaptiveHistory.enabled according to the code.

I heavily disagree.

This is not a bug to me, this is perfect behavior.

+1 Keep as is.

(In reply to Will from comment #30)

Some constructive feedback on this one: please consider making this optional because, for example, I'm used to just typing "i", which makes "instagram.com/" autofill, then I simply hit the right arrow key on my keyboard and from there I can very quickly and efficiently get to any instagram account I want. But now when I hit "i", "instagram.com/explore/tags/formula1/" is autofilling... For me, this is a huge downgrade in speed and UX efficiency.

I have the same concerns. I like the concept of adaptive history, but would prefer that domains shouldn't be excluded. That is, they should be first if they're visited most often.

Why is this suddenly the default behavior? There is nothing in this bug report that indicates or explains why it should be the default.

The default behavior for most users is unchanged. Currently only nightly and potentially experiments will have the behavior changed via browser.urlbar.autoFill.adaptiveHistory.enabled to evaluate.

https://searchfox.org/mozilla-central/rev/6d396b2abda4371f54251e9de8dc790deab706fc/browser/app/profile/firefox.js#360-365

(In reply to Ed Lee :Mardak from comment #35)

The default behavior for most users is unchanged. Currently only nightly and potentially experiments will have the behavior changed via browser.urlbar.autoFill.adaptiveHistory.enabled to evaluate.

https://searchfox.org/mozilla-central/rev/6d396b2abda4371f54251e9de8dc790deab706fc/browser/app/profile/firefox.js#360-365

Thanks, that's good to know. I'm a nightly user, does it help you if I keep it enabled? I don't like it but If it helps you make the feature better I can keep it for now.

Apologies for a redundant comment: Similar to Will this disrupts my muscle memory, e.g. I'm used to typing e for the English Wikipedia front page. For now I've toggled browser.urlbar.autoFill.adaptiveHistory.enabled to false. Maybe statistics will show that I'm in a small minority that use it this way, but I expect this is going to be frustrating for other people as well, who may not know where to look for a pref.

Also, I don't use tab-to-search but for those that do use it, has this had any negative effect on that?

(In reply to Ori Avtalion (salty-horse) from comment #33)

I have the same concerns. I like the concept of adaptive history, but would prefer that domains shouldn't be excluded. That is, they should be first if they're visited most often.

I don't think domains are excluded, and they're mostly showing up first for me. Probably the base domain you're trying to reach just has lower frecency than some particular location on the same domain. Personally, I'm gonna disable this pref for now since I want to visit domains radically more often than I want to visit particular URLs. I haven't had time to look at the patches so forgive me if you already did this lol. But if it was weighted more heavily toward origins that could resolve it for me. Like, maybe origin frecency could be worth 2 or 3 times as much as URL frecency. (I guess that weighting ratio could be a user preference, eh)

But actually, even that might not help in some conditions. The first thing I noticed was that when I try to reach youtube.com I was getting https://www.youtube.com/watch?v=*&list=*&index=1%24 because there must be thousands of entries. That's a sort of fake URL generator I made for an alarm clock. So although the frecency is insanely higher than youtube.com (because it launches every day), it's never a URL that I want to visit manually. That's an extreme example but I'm guessing there might be other similar flukes. I guess that could be mitigated if there was a way to exclude results without deleting them from the db.

(In reply to Shane Hughes [:aminomancer] from comment #39)

(In reply to Ori Avtalion (salty-horse) from comment #33)
I don't think domains are excluded, and they're mostly showing up first for me.

If domains aren't excluded, then unless "training" started yesterday, the behavior is very odd. When I type "twitter" the top result is a specific user's page, that I rarely visit, while "https://twitter.com/home" is missing, and I visit it several times a day.

(In reply to Adam Gashlin (he/him) [:agashlin] (ex-moco) from comment #37)

Apologies for a redundant comment: Similar to Will this disrupts my muscle memory, e.g. I'm used to typing e for the English Wikipedia front page. For now I've toggled browser.urlbar.autoFill.adaptiveHistory.enabled to false. Maybe statistics will show that I'm in a small minority that use it this way, but I expect this is going to be frustrating for other people as well, who may not know where to look for a pref.

Do you get the previous behavior back if you bookmark the front page of English Wikipedia?

(In reply to George Sofianos from comment #34)

Why is this suddenly the default behavior? There is nothing in this bug report that indicates or explains why it should be the default.

I filed a separate bug that was merged into this one, so I'll give you my reasoning, though I cannot speak for Robert Barrett:

When I type 'e' in the search bar, it should populate with literally any bookmark I have that has an 'e' in it before it should populate with Firefox's wrong guesses about what other, non-bookmarked, sites I might want to visit. If I wanted to visit them a lot, they'd be bookmarked.

(In reply to Ori Avtalion (salty-horse) from comment #40)

If domains aren't excluded, then unless "training" started yesterday, the behavior is very odd. When I type "twitter" the top result is a specific user's page, that I rarely visit, while "https://twitter.com/home" is missing, and I visit it several times a day.

That's interesting, did you type the full twitter string or just part of it? Also, if you visit twitter.com (no path) and then repeat the test, does that change the autofill result? (Keep in mind urlbar results are based on frequency but also on date of last visit. So even if a particular URL has a much higher frequency, if a different URL has a higher recency then it may be shown instead)

(In reply to soren.nissen+mozilla from comment #41)

Do you get the previous behavior back if you bookmark the front page of English Wikipedia?

No, though now (with or without a bookmark) it's filling e to https://en.wikipedia.org/wiki/Main_Page which is what en.wikipedia.org redirects to, maybe I've since selected it from the dropdown (see below)? Before it was filling a specific page, https://en.wikipedia.org/w/index.php?title=%F0%9F%84%81, which itself is a redirect, and not one I used frequently or recently, maybe it just sorts early or oddly?

I spotted a few other cases of bookmarked, frequent sites that don't autofill with adaptiveHistory.enabled, in favor of rarer sites. For instance lo autofills one site with 50 visits (https://love2d.org/wiki/love.graphics, not bookmarked) instead of previous autofill of a more recent site (https://lobste.rs/, bookmarked) with 892 visits (as counted by History, Page Info counts 458 and 2,969 respectively, I think that covers the whole domain). If I use the down arrow key to select the preferred site once from where they appear in the dropdown below "Firefox Suggest", then they will autofill the next time, is that the "adaptive" part?

This autofill behavior may well be better, but it's at least been a surprising change, and maybe the ranking is off somehow.

(In reply to Shane Hughes [:aminomancer] from comment #43)

(In reply to Ori Avtalion (salty-horse) from comment #40)
That's interesting, did you type the full twitter string or just part of it? Also, if you visit twitter.com (no path) and then repeat the test, does that change the autofill result? (Keep in mind urlbar results are based on frequency but also on date of last visit. So even if a particular URL has a much higher frequency, if a different URL has a higher recency then it may be shown instead)

I typed the full domain "twitter.com". Throughout typing, it never offers the base domain. The address bar is prefilled the last-visited user page. In the dropdown, the top offering is another user page, and then it's followed by pages of specific tweets.

I am experimenting similar problems as others. ex:

• Typing 'a' leads me to https://addons.mozilla.org/fr/firefox/addon/bmo-share/ which is a recent extension I published, I visited it frequently recently because I just created it, but it used to go to amazon which I have been visiting daily for a decade
• Typing 'am' still doesn't lead me to amazon.fr, it leads me to ameli.fr (social security site) which I visit only a few times a year
• Typing 'ama' finally proposes amazon.fr

Generally speaking, it seems to me that the domain name in the autocompletion should have more weight and I also don't understand why some specific urls which I haven't visited in months are proposed as a first choice (for example typing 'b' autocompletes to https://bugzilla.mozilla.org/show_bug.cgi?id=1706678 which is not a bug I visit frequently today, going straight to bugzilla or a bug I visited more recently would be more useful to me). I think that if you type only one letter, my expectation is to autocomplete on the domain name first, not a random letter in the url or the subdomain. Typing m used to give me mozilla.org, now It gives me mana.mozilla.org. I visit mozilla.org a lot more often than our mana page. Maybe because https://mozilla.org redirects to https://www.mozilla.org this has lower priority than mana.mozilla.org with the newer algorithm, same with amazon.fr which goes to www.amazon.fr.

I found this breaking all of the muscle memory I had up to this point in a frustrating way. Thankfully this is possible to disable. I think it needs much more work before enabling by default (if ever). And I hope browser.urlbar.autoFill.adaptiveHistory.enabled option will remain available going forward so I never see this behavior again.

For context my usage of URL autocomplete is the following:

1. g to get github.com
2. first characters of the organization or account
3. first characters of the repo
4. optionally issues/discussions/etc part of the URL

Sometimes 2+ is replaced with Up/Down arrows on keyboard for recent URLs. As you can imagine having the whole URL inserted breaks the experience completely and requires a lot of typing to erase unnecessary parts of the URL.

We'd like to thank everyone for the feedback, this is something we'd really like to ship users that asked for the behavior for years, but of course it must not be disruptive for everyone else.
We identified the problem with the matching and training, and have some plans to address those, thus we're going to disable the experimental feature in Nightly until those are addressed, and hopefully the next time you'll enjoy it!
Please stay tuned, and thank you again for immediate feedback.

I just landed bug 1769783, which disables this new autofill by default. It will be in whichever Nightly is built after it's merged to mozilla-central. We'll work on improving the matching logic in bug 1769585.

If you're willing to re-enable browser.urlbar.autoFill.adaptiveHistory.enabled on the latest Nightly and try out the feature again, please do so and let us know how well it works for you. We'd appreciate it. We've made some improvements to the feature over the past week, and we'd like to gather feedback. It's still disabled by default, but we'd like to re-enable it soon.

The main improvements are:

• The full search string stored in an adaptive history record now must be typed in order to autofill its related URL. Previously, any substring that the search string started with could be typed. This should cut down on the number of times where you only type two or three characters and a full URL is autofilled. (It can still happen when the search string in the adaptive history record is only two or three chars long.)
• The feature should adapt a little better than before (over a period of days). Triggering and picking an adaptive history autofill will keep it autofilling for you in the future, and if you never pick it again, it will stop autofilling more quickly than before (30 days instead of 90).

Though I have faith the team will be able to improve this, it's not possible to perfect this (even Chrome is still struggling with this, as you can see from the attached screenshot), so please add a "Remove suggestion" button like Chrome has! People have been asking for this button anyways...

(apologies if this belongs in bug 1769546)

(In reply to Will from comment #51)

Created attachment 9279913 [details]
remove suggestion button.png

Though I have faith the team will be able to improve this, it's not possible to perfect this (even Chrome is still struggling with this, as you can see from the attached screenshot), so please add a "Remove suggestion" button like Chrome has! People have been asking for this button anyways...

(apologies if this belongs in bug 1769546)

Do you mind filing a new bug for that? I can work on adding a button but I need a bug to attach the patch to. Thanks

(In reply to Shane Hughes [:aminomancer] from comment #52)

Do you mind filing a new bug for that? I can work on adding a button but I need a bug to attach the patch to. Thanks

We have at least one bug on file already, bug 675818, but Shane, please don't work on it until/unless we get a green light from Product, which they have not provided for this.

But please explain to Product why we really do need that "Remove suggestion" button now because of this "autofill entire URL's" change, please don't just ask for the button without context of this issue because they'd be more likely to say no.

Moving over some bugs that were blocked by bug 1466233, which I just marked as a dupe of this bug. I'm not sure all of these are really blocked by adaptive history autofill.

(In reply to posilejte.spam from comment #19)

In my opinion it should only autofill the entire URL when the user has bookmarked itand has disabled all suggestions except for bookmarks.

As someone who has intentionally done exactly that (disabled all suggestions except for bookmarks), I'd be fine with this.

I want only bookmarked URLs to be presented as suggestions, without exception. Enabling "adaptiveHistory" does make the situation better than it was (i.e. bookmarks seem to get a boost relative to domains), but it still leaves me sometimes getting suggestions (i.e. domains) that I don't want. As things stand, there doesn't seem to be a way to get the required behaviour: unbookmarked URLs creep in to autocomplete unless suggestions are completely disabled.