User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0

Steps to reproduce:

Enable DNS over HTTPS

Actual results:

My pre-configured hosts file should be read and used

Expected results:

Said host file is not.

This problem was already brought up in bug 1453207, but was closed as a wontfix with essentially no discussion. With DoH/T now on the cusp of release and indeed having plans of turning it on by default, I feel the need to open another discussion about this feature.

As much as I agree with turning DoH/DoT on by default, there are many users concerned about an app as important as the browser suddenly hijacking a carefully configured DNS setup. As far as I can tell, the current setup allows you to pass through "excluded domains" to the OS resolver, but that requires the user to copy/paste every domain in their hosts file which for some people is quite the list. The feature can be turned off, but not only is that reactionary (and some might not notice right away), some of those users WANT this feature, and I haven't found a suitable way to get it at the OS level on Windows yet.

If you're going to act as a DNS resolver, especially if you're going to take over DNS queries without asking the user at some point, you should at least try to act as a competent one and support a basic feature.

I think there's another, more subtle point here. The usual answer to these questions seems to be "here's how you can disable TRR".

But some of the newer browser features are actually dependent on TRR. Even if I set up a resolver that uses Cloudflare, I won't get ESNI in Firefox because of what looks like an arbitrary decision.

So it seems that users who can't use TRR because of limitations like not being able to read hosts, will actually be missing on new features.

The feature can be turned off, but not only is that reactionary (and some might not notice right away), some of those users WANT this feature, and I haven't found a suitable way to get it at the OS level on Windows yet.

Hi Nathan, dnscrypt-proxy 2 supports DoH in addition to DNSCrypt. You can configure it to use both protocols or only which one you want, although keep in mind some servers only support one protocol or the other when selecting which servers to use or blacklist or whitelist. You don't need to regularly update it, and it automatically updates the resolvers list regularly so you don't have dead resolvers.

For the record I agree that any browser implementing its own DNS lookup should read the system's hosts file, and those domains should not be queried. I myself use the MVPS hosts file plus a handful of domains I've added manually.

As Firefox 69 enables DoH/TRR by default for a large number of users and hosts files are one of the most popular methods to block trackers or other unwanted hostnames that may not be present on the Disconnect list used by Firefox, it would be great if this request got more traction/attention.

Essentially, DoH in its current form circumvents the user's choice not to connect to specific hosts.
This is particularly problematic for Firefox variants that don't have about:config (e.g. Focus aka Klar) or users who have more than one Firefox profile.

I see the following possible solutions, please feel free to expand:

1. Currently, DoH can be disabled by resolvers by returning NXDOMAIN for use-application-dns.net. Unfortunately, hosts files can not return NXDOMAIN, only unroutable or otherwise non-functional IPs. If this condition were extended to also include a specific IP, e.g. 0.0.0.0, it would be trivial for anyone who maintains a hosts file to include such an entry for use-application-dns.net.
   Pro: Should be simple to implement, both on Firefox side and for hosts lists maintainers.
   Contra: Disables DoH.

2. Parse the hosts file and include it as part of the "Tracking Protection" feature. A potential issue here might be that the location of the hosts file is not uniform for all systems (but at least the format is, AFAIK).
   Pro: No changes required to hosts files, compatible with DoH.
   Contra: Requires a parser for hosts files and might make some invalid assumptions about their location. Also, changes are probably not propagated to Firefox immediately unless the hosts file is watched by inotify et al.

3. Expose a UI to add custom blacklists to the "Tracking Protection" feature, either via a URL or file location. This is more or less a variation on solution number 2. Pro/Contra should be identical, except that no assumptions about the underlying OS have to be made.

4. Introduce some form of system-wide DoH server list (think resolv.conf, but for DoH resolver URLs). Together with some local resolver that handles the actual blacklist, this would save admins the effort of having to modify all Firefox profiles.
   Pro: Compatible with DoH, encourages DoH resolver decentralization
   Contra: Requires users to set up & maintain DoH resolvers, invents a new de-facto standard configuration file or environment variable, somewhat UNIX specific (probably doesn't have to be, $HOME should exist on all OS but Android/iOS).

I think that the easiest solution by far, and the one that makes the most sense to implement, would be to simply parse the user's host file for domains, and append the domains it finds to the end of the network.trr.excluded-domains list for the session.

This way, Firefox doesn't have to include any functionality it doesn't already provide: the domains are send to the local DNS which uses the hosts file. On top of that, if you make it a session-only list of domains, restarting Firefox would reload the hosts file if you make changes to it, at the cost of a slight hit to startup time.

Should this be treated as a security issue? A complaint on HN today would suggest it should be.

Dragana, would you also mind providing your current thinking on this?

Previous decision was not to read hosts files, because there is a way to black list some hosts for TRR and Firefox will use a native resolver for them. The host in the hosts file should be in the TRR's blacklist as well. Maybe this consequence should have been communicated it better in the past. We will also have a better interface for blacklisting at some point, I am not sure about the time line.

do not consider this a security issue.

I see doing it automatically may be useful, but at the same time will be a performance issue. If we decide to do it I am not sure of a time frame.

We talked about this in the triage meeting and came to the conclusion that this is a privacy concern and because we do use the sec rating system for privacy concerns as well as security it merits a sec-low.

I'll start implementing this ASAP.

Firefox should look at the hosts file before embarking on TRR resolution. That is where users place tweaks for local network configurations, test configurations, and black lists.

network.trr.excluded-domains is not a replacement for the hosts file (e.g., cannot hold tens of thousands of entries, as contained in many blacklist configurations).

Fixed in bug 1616252.