User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0

Steps to reproduce:

Coming off this bug report we got: https://github.com/darkreader/darkreader/issues/5937

It seems like HTTPS-Only mode rewrite fetch request from http:// -> https:// scheme, however this causes to actually fetch a resource which doesn't exist, as this particular site doesn't have HTTPS enabled/implemented.

It also happens in normal browser context from a site, but because to stay in the same context Il stick to a background-script doing this fetch request as it's recommended for extensions.

Actual results:

• Background script execute a fetch request to a non HTTPS-scheme url(because it was requested that way)
• The fetch requests returns a NetworkError.

Expected results:

• Background script execute a fetch request to a non HTTPS-scheme url(because it was requested that way)
• The fetch requests returns the content of the site.

The Bugbug bot thinks this bug should belong to the 'Core::DOM: Networking' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

This bug doesn't make sense with how HTTPS-only mode works. If you opened the site at all then none of the resources should have been upgraded.

First, you can't possibly be using Firefox 78 because this feature didn't exist then. Knowing what version you actually used would help.

I am not sure what site you're talking about exactly, please correct me if I've made a wrong assumption. The github link in this bug is not the problem, that is an issue describing a problem with http://kb.mozillazine.org/Knowledge_Base

I have no problem going to http://kb.mozillazine.org/Knowledge_Base with HTTPS-only mode in Firefox Release (89) or Nightly (91). In order to open that site at all I got the HTTPS-only mode warning page that the site was unavailable under a secure (https) scheme. I clicked the override button which opened the mozillazine page using insecure http. The site opens fine. Reload is fine. If you can see the site at all then we've disabled the https-only mode for that site, so it would not apply to any sub-resources including styles.

Is there anything more you can add to the steps to help us reproduce this problem?

First of all, sorry I wasn't really being clear in my bug.

This bug doesn't make sense with how HTTPS-only mode works. If you opened the site at all then none of the resources should have been upgraded.

That's correct, within the context of the site it doesn't upgrade the resource. However as we are being limited by CORS and and other factors as a extension we have to make a fetch request to this resource from a background-scripts which happily will upgrade the resource to a HTTPS endpoint which doesn't exist.

So simple terms:
Correct behavior is you do the fetch resource from site's context(which we can't due to CORS and others).
Incorrect behavior by upgrading the resource if you fetch the resource from a background script of an extension.

First, you can't possibly be using Firefox 78 because this feature didn't exist then.

Ah, must've been privacy.resistFingerprinting that's spoofing the user agent to Firefox 78, but I've noticed it in Firefox 88 and now with the recent release also in Firefox 89

Is there anything more you can add to the steps to help us reproduce this problem?

I've attached a test-case.zip in the root comment, which is a simple simple extension to fetch this resource, whereby you can observe that the resource is being upgraded and then will error out.

It's not really the same behavior as how this works in an content-script/background-script communication. Because in this test-case.zip a hardcoded fetch is being used. Whereby otherwise such requests will come from the content script which may or may not come from a non-HTTPS site and then the background-script will fetch the resource, which may or may not come from a non-HTTPS site.

Shane, and/or Rob, now that we know what the problem is - any chance you can help investigate/fix this problem?

As agreed, if https-only mode is on, then we upgrade all resource loads. If a site is exempt however, then I don't think that we use the correct TriggeringPrincipal, or a triggeringPricnipal that would allow us to infer where the request ends up being used, for the background script request.

There is a related issue and discussion about https-upgrades in the downloads extension API (bug 1685862). There was an implementation plan sketched at bug 1685862, with the suggestion to do something in the DownloadCore implementation.

Might it be feasible to address both issues at once by checking for site permissions for the specific host when the requestor is an extension principal that has extension/host permissions to access the site? That would cover at least this bug, and cover common cases of bug 1685862 (common = use of downloads.download where the extension has host permissions; uncommon = use of downloads.download API without host permissions).

nsHTTPSOnlyUtils::ShouldUpgradeRequest should just exempt requests where the loading principal has an addonPolicy. That would fix a slew of problems.

this can then be refined via followups when there is time.

Would that be an acceptable approach?

Yeah, that's an acceptable approach. Please note that we would need that carve out within ShouldUpgradeRequest and also within ShouldUpgradeHttpsFirstRequest, so probably some helper function to be called from within the two is the way to go - thanks!

For users, are there any known workarounds other than disabling HTTPS Only mode?

(In reply to Daniel from comment #8)

For users, are there any known workarounds other than disabling HTTPS Only mode?

Did you find a workaround that did not involve disabling HTTPS Only mode?