User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:73.0) Gecko/20100101 Firefox/73.0

Steps to reproduce:

Simply type "w" into the empty URL bar and I'm sure Firefox autofills a website starting with "www" (in my case, it's www.reddit.com). This is not right and Firefox should be more intelligent than this. The URL Firefox is autofilling (www.reddit.com) doesn't start with a "w", it starts with an "r", it should not be suggesting www.reddit.com when I've only typed "w". When a user types "w" into the URL bar, they should either be getting URL's that start with "w" (such as walmart.com or wikipedia.org etc.) or site titles that start with "w" like "Wikipedia, the free encyclopedia".

Expected results:

screenshot attached I've what I'm seeing

I forgot to add: Firefox already does something like this for "h". When I type "h" into the empty URL bar, I do NOT get autofill for a URL that starts with "https://" or "http://" , even though basically every URL starts with that. This behaviour should be extended to "www."

That's debatable. It's easy to imagine people typing "w" in order to to visit www.facebook.com.

(In reply to Drew Willcoxon :adw from comment #2)

That's debatable. It's easy to imagine people typing "w" in order to to visit www.facebook.com.

Yes and it's been debated a lot over the years but there does have to be a line drawn at some point: what is correct and what is incorrect. If someone got used to typing "w" to go to facebook, that's wrong because it makes no sense. It was an unintended quirk that they user got used to. "Facebook" doesn't even have a "w" in it! If they want to go to facebook, they should type "f", this is just as fast/efficient as typing "w" AND it actually makes sense, that is how Firefox should work. If we let scenarios like the one you mentioned dictate Firefox, Firefox would be a complete mess, it would be madness. I'm well aware that people really despise changes but that shouldn't discourage us from improving Firefox in ways like this.

Also, we have to remember that there are websites that start with "w" (walmart.com) that people are wanting and expecting to autofill when they type "w", but instead they're getting autofills such as www.google.com, www.facebook.com etc. And I'm sure those websites aren't happy about that either!

Also I should change the title of this bug because, if someone types 2 double-ues ("ww") into the URL bar, of course "www.facebook" etc. should autofill, unless there are URL's in their history/bookmarks such as wwe.com or wwf.ca. Then those should autofill before www.facebook.com, www.google.com etc.

This should also apply to Fenix (same with typing "h" resulting in "https://example.com" being autofilled. Firefox desktop already fixed the "h" scenario a while ago though)

I discussed this bug with Drew yesterday and its scope is larger than expected. The moz_origins table we use for autofill splits URLs up into prefix and host, where prefix is the scheme, a colon, and possibly two slashes, and host is the rest, possibly including www.. The autofill SQL query searches the host field. This is how we don't search for http when the user types "h".

Drew initially suggested modifying the SQL query to add something like this:

WHERE (host BETWEEN :searchString AND :searchString || X'FFFF') OR
      (
        substr(host, 0, 4) = 'www.' AND
        substr(host, 4) BETWEEN :searchString AND :searchString || X'FFFF'
      )

But he pointed out this would simply search for www.site.com and site.com for every autofill query. This means www.site.com might still be autofilled. To not search www., we'd need to strip www. from the host field in moz_origins. That means typing www.mozilla.org would never autofill www.mozilla.org. Instead, you’d have to type mozilla.org to autofill mozilla.org instead. This seems undesirable.

There's probably a middle path with a more specific implementation. For example, if one types "w”, we could prefer matching domains that start with w after stripping www. from every domain, but fall back to matching www.. This needs some discussion and an engineering definition. I'm un-assigning myself, pending discussion.

Marco, do you have any thoughts on how to handle this?

Should we retitle this just to be clearer? Maybe "(URL bar autofill) domains/origins should always be prioritized over "www" when user types "w".

Like I said in comment 4, if someone types 2 double-ues (and they've never visited a site that starts with 2 or 3 w's, such as wwe.com), they should continue to get autofills like www.google.com or www.facebook.com etc.

(In reply to Harry Twyford [:harry] from comment #6)

I discussed this bug with Drew yesterday and its scope is larger than expected. The moz_origins table we use for autofill splits URLs up into prefix and host, where prefix is the scheme, a colon, and possibly two slashes, and host is the rest, possibly including www..

We have a similar problem in bug 843357, we should start investigating the removal of www from both, but that needs an analysis to understand possible drawbacks. That discussion may also depend on the outcome of the stripWww pref. Mostly we should ensure in cases where those 2 origins differ, the user has still ways to make a choice in the urlbar.
This requires checking how rev_host and moz_origins are used today across the codebase, and marking down every behavior that'd change. If those changes are acceptable (for example grouping by host in history UI without www would be acceptable, forcing the user to go the non-www version when it doesn't exist wouldn't be acceptable) then we could proceed with a schema migration.
It requires some design effort but it should be feasible, and then we'd have all the data we need to decide.

There's probably a middle path with a more specific implementation. For example, if one types "w”, we could prefer matching domains that start with w after stripping www. from every domain, but fall back to matching www.. This needs some discussion and an engineering definition. I'm un-assigning myself, pending discussion.

I'm not sure how you'd do that, in particular the stripping for every domain would be far too expensive, we rely on the data store capabilities here, we can only match at the beginning of the url/origin string, not in the middle of it.
I wonder if bug 1466233 would help here, but we should change the input history table contents, in particular split "scheme://www." into a separate column. That is sort of planned anyway if we want to use that data for autofill.

Bumping up the points value here, keeping it as a P3 but requires some deeper refactoring, so not immediately actionable.
Thanks for the initial investigation.

Just wondering something about this issue. Take wikipedia's URL for example: for English, it's "en." subdomain https://en.wikipedia.org/wiki/Main_Page . It's different depending on the language (https://de.wikipedia.org/wiki/Wikipedia:Hauptseite etc.). When you begin working on this issue, please investigate whether it's possible to "ignore" these subdomains (apologies if subdomain is not the correct word here, I mean "en." or "de." or "ru." portion etc.), that way, when the user types "w" into the URL bar, https://en.wikipedia.org/ would autofill. It really does feel like this is how it should behave, I hope it's possible.

Sorry, I forgot to add: another benefit of this would be faster/more efficient tab-to-search: type "w" (wikipedia autofills), hit "Tab", then search!

Oh wow, I'm really worry for posting yet again because I know this one isn't a high priority but I had a weird config issue and I noticed a bug that proves this issue really does need to get fixed (tab-to-search wasn't working for me at all, it's working now). Now, I type "w", "www.google.com/" autofills. Then I hit "Tab" and Wikipedia tab to search comes up! That's a really confusing issue, see attached screenshots

(In reply to Will from comment #9)

that way, when the user types "w" into the URL bar, https://en.wikipedia.org/ would autofill. It really does feel like this is how it should behave, I hope it's possible.

Technically it would not be impossible,there's a few risks to consider because urls can be built in very fancy ways. It could probably be done for a subset of cases (lang-code.domain), but it's even more complex than www. One step at a time.

(In reply to Will from comment #11)

Oh wow, I'm really worry for posting yet again because I know this one isn't a high priority but I had a weird config issue and I noticed a bug that proves this issue really does need to get fixed (tab-to-search wasn't working for me at all, it's working now). Now, I type "w", "www.google.com/" autofills. Then I hit "Tab" and Wikipedia tab to search comes up! That's a really confusing issue, see attached screenshots

Hm, that looks indeed strange, but unrelated to this bug.
I suspect it's a regression from recent results composition work, please file a separate bug so we can fix it.

But I think it IS related to this bug. If both of these issues are fixable (comment #0 and comment #9 too), it would autofill "https://en.wikipedia.org/" when you enter "w". That would solve the issue I brought up in comment #11

This bug is a lot more complex to fix and we don't have the time to do it now. In comment 11 the tab to search result for wikipedia should not be there, because wikipedia was not autofilled, we should fix that bug and still have this open for the future.

About the comment #11 issue, maybe ask the people that worked on tab-to-search how this is supposed to work. I assumed they purposely designed it so that, when the user types "w", wikipedia comes up (because it's one of the default search engines included in Firefox).

(In reply to Will from comment #15)

About the comment #11 issue, maybe ask the people that worked on tab-to-search how this is supposed to work. I assumed they purposely designed it so that, when the user types "w", wikipedia comes up (because it's one of the default search engines included in Firefox).

It's not supposed to work that way. We were specifically concerned about inundating the user with too many tab-to-search results. We're only supposed to show a tab-to-search result if the engine domain is autofilled. It would be annoying if we showed a Google tab-to-search result every time "g" was typed, but that appears to be almost what we're doing. I can reproduce in a different scenario: when an autofilled site starts with "g", we show a tab-to-search result for Google Drive. Weirdly, if I disable my Google Drive engine, neither Google nor Google Maps (which I have installed as an OpenSearch engine!) shows a tab-to-search result. It's not clear what's broken here.

I ran mozregression and the Wikipedia issue was sort of reproducible on the 2020-10-01 build when tab-to-search was introduced. A key difference being that we didn't yet show a Wikipedia tab-to-search result when "wikipedia.org" was autofilled, because the engine was actually for <locale>.wikipedia.org. I could reproduce the issue after installing an OpenSearch engine that pointed directly to wikipedia.org.

Something weird is happening with the muxer here. I'll file and fix.

(In reply to Harry Twyford [:harry] from comment #16)

I'll file and fix.

Bug 1713662.

This affects the Wikipedia tab to search. Testes on nightly 92.0a1
when i type "w" it does not give me the wikipedia tab to search option.

not sure if another bug is needed, please let me know and i will open one.

It's a little confusing, but you have to get en.wikipedia.org (or whatever subdomain matches your Wikipedia search engine) to start autofilling, and then when you start typing "wikipedia", you should get a TTS result. If instead you get wikipedia.org (without the subdomain) to start autofilling, that won't trigger it because there's no wikipedia.org search engine.

i.e., it should work with these steps on a new profile on Nightly:

1. Type en.wikipedia.org and hit enter (you'll also see the TTS result for en.wikipedia.org during this, just ignore it)
2. Now start typing wikipedia

You should see the Wikipedia TTS at that point.

I think the biggest/best reason to fix this sooner rather than later is because URL's that start with "w" (walmart.com etc.) are being penalized by this behavior. Wikipedia seems to be trickier because of the language subdomain en.wikipedia.org (apologies if subdomain is not the correct term, I think you know what I mean). Though https://www.wikipedia.org also exists!

yes, that's exactly why we should fix it, penalizing urls starting with w to favor www. doesn't make sense.

I wanted to ask before bothering to file this but is this the same issue? (typing "m" autofills "mobile.twitter.com/" , but in my opinion this is incorrect/undesirable. It should autofill "macrumors.com/"). Probably more relevant on Fenix but it's still an issue on desktop Firefox, as you can see in the screenshot

Potentially it's the same bug, yes.
Of course we cannot handle any possible subdomain. I briefly looked around and found a few articles that points out at www, mail, webmail, forum, m, blog, shop, forums, wiki as very common.
We likely don't want to dismiss a subdomain that substantially changes contents of the page, in that sense www and m (potentially mobile) seem to make sense, they just change nothing or the presentation.
The other thing we may want to do is lowering the score if the matching is in the subdomain part, though finding the right ratio may not be trivial.

Thank you for investigating this! Another one that might cause issue is tumblr accounts (example https://siriuslymeg.tumblr.com/ . A user should be able to type "s" and get "siriuslymeg.tumblr.com/" to autofill)

Sorry for the back to back comments but I was just looking around and noticed Chrome people are working on "rich autocompletion" , does Firefox have an equivalent tracking bug for this? https://bugs.chromium.org/p/chromium/issues/detail?id=1062446

that's a bit off topic, but no, so far we don't plan to provide "in-the-middle" autofill. We had something similar in the past, and it was very confusing for the users, and hard to get it right in code. There is a reason if the Chrome fix is still pending after 2 years, it's not trivial to get that working in a non confusing way for the user. If their new solution works better than what we had we'll surely evaluate it as an idea.

The reason I brought up rich autocompletion in comment 25 is because if Firefox had that, it could make implementing this bug (1610718) possible or at least easier. Of course I don't know exactly how this bug (1610718) would work if implemented 'as is' but there does seem to be a few valid/legit concerns (see comments 23 and 24). Does anyone know if there's a bug or something for Firefox to investigate rich autocompletion? Using rich autocompletion in Chrome Canary, I think it's definitely worth exploring and experimenting but some of the UI/UX decisions the Chrome team have went with are poor, but that doesn't mean the entire idea of rich autocompletion should be thrown in the garbage.

(In reply to Will from comment #27)

The reason I brought up rich autocompletion in comment 25 is because if Firefox had that, it could make implementing this bug (1610718) possible or at least easier. Of course I don't know exactly how this bug (1610718) would work if implemented 'as is' but there does seem to be a few valid/legit concerns (see comments 23 and 24). Does anyone know if there's a bug or something for Firefox to investigate rich autocompletion?

We have investigated it in the past, but the UX for it seems a bit confusing, we're not convinced it's a net benefit for users in the current status.
We don't need that to fix this bug anyway, it can be fixed on its own.