User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:89.0) Gecko/20100101 Firefox/89.0

Steps to reproduce:

I'm trying to help a developer of an add-on for Firefox debug an issue. The add-on is a simple userscript manager built around Firefox's userScript API - it doesn't inject scripts into the page or the content context like the popular userscript managers do.

The problem is that simple scripts using XMLHttpRequest() fail if those requests are cross-origin, even if those requests are allowed by CORS.

I have tested this with both the developer's own extension and the example web extension "user-script-register" on the MDN Github. Both have this problem. The developer confirmed this problem exists as well.

So while it is unlikely to be a problem caused by the specific extension, here is the downstream bug report: https://github.com/erosman/support/issues/340

Here is a simple user script which fails:

// ==UserScript==
// @name     XMLHttpRequest test
// @version  1
// @grant    none
// @match    https://example.org/*
// ==/UserScript==

function reqListener () {
  console.log("response received");
}

var oReq = new XMLHttpRequest();
oReq.addEventListener("load", reqListener);
oReq.open("GET", "https://httpbin.org/status/200");
oReq.send();

Note that httpbin.org has Access-Control-Allow-Origin: * set, so this is not an issue caused by CORS blocking the request (despite the strange error described below).

Steps to reproduce:

1. Register this script with the userScripts API
2. Go to https://example.org
3. Check the devtools console.

Actual results:

Nothing happens in the devtools console.

On Windows 10 + Firefox 89, I see the error described here in the browser console: https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS/Errors/CORSOriginHeaderNotAdded

I also see the following:

Error: Got a request https://httpbin.org/status/200 without a browsingContextID set

The latter error is the only error I see in Firefox 89 + Linux.

Expected results:

No errors should be printed, the request should be sent, and "response received" should be printed when the callback fires.

Note that content scripts, as injected by Greasemonkey (for example), do not have any issue. Similarly, I can run the same code in the console or directly in the page in script tags.

The developer of the extension thinks blocking all cross-origin requests may be intended behavior. I find this very unlikely, but on the off chance that it's true, it would be extremely useful to add this to the documentation.

The Bugbug bot thinks this bug should belong to the 'WebExtensions::Request Handling' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Likely a duplicate of bug 1605197. Interestingly, a user script sandbox doesn't have host permissions (behind the scenes it has an expanded principal consisting of the page's principal only).

(In reply to Rob Wu [:robwu] from comment #2)
Thanks for the reply.

I'm confused about the situation here. As I understand it the current behavior with content scripts is in line with the docs, which say

Content scripts get the same cross-domain privileges as the rest of the extension: so if the extension has requested cross-domain access for a domain using the permissions key in manifest.json, then its content scripts get access that domain as well.

This is accomplished by exposing more privileged XHR and fetch instances in the content script, which has the side-effect of not setting the Origin and Referer headers like a request from the page itself would; this is often preferable to prevent the request from revealing its cross-origin nature.

In other words, if an extension doesn't have cross-domain access to a domain, it doesn't get to do cross-origin requests to it. That's in line with what's reported in bug 1605197, where you say that if an "extension does not have host permissions in a content script", you get an error. As I understand it, fixing that bug would change the documented behavior.

This issue does not affect most user script managers (e.g. Greasemonkey) because they already request host permission for all URLs. That makes it seem to me like this issue is either (a) the user script sandbox should have host permissions when the extension does (similar to content scripts), or (b) the documentation for the user script sandbox should be updated to clarify that the sandbox does not have host permissions, and therefore cannot do cross-site requests (in line with content scripts without host permissions not being allowed to do them).

Maybe you're saying that the documented behavior is undesirable and we want sandboxed scripts to have access to cross-origin requests (with the origin header set) in the short term, while banning cross-origin requests in the long term (see below)?

Can I ask a few follow up questions as well?

1. Reading bug 1578405 it seems that the future is banning cross-origin requests from content scripts entirely. For better or worse, popular user script managers (e.g. Greasemonkey) have settled on using content scripts as the sandbox for user scripts. Does this change mean that user script authors should only be relying on APIs provided by their user script manager to go cross origin, for future proofing? Will ordinary XMLHttpRequest be impossible in the future?

2. Is there an appropriate workaround for an extension author in the mean time? There doesn't seem to have been any real movement on that bug since it was created over 2 years ago.

Hello,

I reproduced the issue on the latest Nightly (91.0a1/20210609093513), Beta (90.0b5/20210608185546) and Release (89.0/20210527174632) under Windows 10 x64.

Tested both Greasemonkey and user-script-register add-ons with the provided user script, with the following results:

• Greasemonkey – on all tested versions, the “response received” message is logged in the browser console. None of the mentioned errors are displayed in the console.

• user-script-register – on none of tested versions did the “response received” message get logged in the console.
  Additionally, the “Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://httpbin.org/status/200. (Reason: CORS header ‘Origin’ cannot be added).“ error message was displayed in the console for all tested versions.
  I also got the “Error: Got a request https://httpbin.org/status/200 without a browsingContextID set” on all versions while having both the browser and web console open at the same time.

For further details, please see the attached screenshot.

Issue investigation

This morning I took a look into this and as I was assuming the issue is related to the fact that at the moment fetch and XMLHttpRequest are imported globals in the userScript sandbox:

    const sandbox = Cu.Sandbox(principal, {
      sandboxName: `User Script registered by ${this.extension.policy.debugName}`,
      ...
      wantGlobalProperties: ["XMLHttpRequest", "fetch"],
      ...
   });
   ...

This is kind of similar to how the content scripts sandbox is configured but:

• the content script sandbox has an expanded principal that includes the extension principal, and so in manifest_version 2 extension that allows the imported fetch and XMLHttpRequest to do cross site requests based on the extension host permission (but in manifest_version 3 this is not going to be allowed anymore)

• the user script sandbox has an expanded principal but it doesn't include the extension principal and so the imported fetch and XMLHttpRequest can't do cross site requests based on the extension host permission (and this part is actually intended, the single userScript is not supposed to silently inherit expanded permission that the userScript manager extension does have)

Nevertheless, I think that importing fetch and XMLHttpRequest global properties was a mistake, because this is making the fetch and XMLHttpRequest to be less capable than the webpage fetch and XMLHttpRequest.

Workaround the issue in the userScript

As an additional proof, the following tweaked version of the userScript works as expected (I tested it on FireMonkey):

...
var oReq = new window.XMLHttpRequest();
...

I haven't tried but I think that FireMonkey may also be able to workaround the issue for all userScripts, by overwriting fetch and XMLHttpRequest with window.fetch and window.XMLHttpRequest from its own apiScript (from inside the browser.userScripts.onBeforeScript listener that is injecting the custom GM APIs), which may worth a try to workaround the issue on Firefox versions that would not yet include the fix on the ExtensionContent.jsm side.

How to allow userScripts to be able to do cross-site requests only allowed by the "extension Host permission"

As an answer to some of the points in comment 3:

to allow a userScript to run cross-site requests (the ones that the website would not be allowed to do but the extension can with the right host permissions) a "userScripts Manager" extension (like FireMonkey) is expected to expose its own custom API to the userScript.

In particular GM.xmlHttpRequest is meant to be the one supporting that use case, and FireMonkey does actually already provide that custom API and so the following tweak on the userScript in comment 0 does also work:

function reqListener () {
  console.log("response received");
}

GM.xmlHttpRequest({
  onload: reqListener,
  method: "GET",
  url: "https://httpbin.org/status/200",
});

The fact that in other userScripts manager fetch and XMLHttpRequest globals supports cross site requests to any third party site is actually a side effect of the fact that in those extensions the userScripts are running as content scripts (and so all userScripts are inheriting extension permissions as a side effect of that).

The userScripts API and its less privileged sandboxes are meant to allow the extension to provide capabilities to userScripts in a more granular way (making the extension able to provide more or less capabilities based on userScripts metadata and the extension own's userScripts permissions management logic).

This is an HAR archive ([1]) which contains 3 requests triggered by the extension using the 3 variation:

• the first one is the one triggered by calling window.XMLHttpRequest from the userScript

  • looking to the Request Headers we can confirm that the request contains "Origin: https://example.org", Sec-Fetch-Mode: cors and Sec-Fetch-Site: cross-site and the request is completed as expected

• the second one is the one triggered by calling GM.xmlHttpRequest

  • looking to the Request Headers we can see that it contains no Origin header, but it does still contain "Sec-Fetch-Mode" and "Sec-Fetch-Site" headers, and the request is completed as expected

• the third and last one is triggered by calling XMLHttpRequest from the userScript and it is the one that is blocked by CORS restrictions

  • looking to the Request Headers we can see that it does not include the Origin header and it does not include "Sec-Fetch-Mode" and "Sec-Fetch-Site" headers neither, and so it is gets blocked

[1]: it can be loaded in the devtools network panel, by choosing "Import HAR File" from the gear menu on the top-right of the network panel.

"Workaround the issue in the userScript" does not appear to work in Nightly any more. Tested with fetch & window.fetch.

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch. (Reason: CORS request did not succeed). Status code: (null).

(In reply to erosman from comment #8)

"Workaround the issue in the userScript" does not appear to work in Nightly any more. Tested with fetch & window.fetch.

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch. (Reason: CORS request did not succeed). Status code: (null).

Could you share a self-contained test case?

I believe that removing the line at https://searchfox.org/mozilla-central/rev/86c98c486f03b598d0f80356b69163fd400ec8aa/toolkit/components/extensions/ExtensionContent.jsm#741 should fix the reported bug, as expected.

Testing window.XMLHttpRequest() & window.fetch()

• Temporary Install & open any page
• Check Developer Tools & Browser Console

Comment 10 works for me on release (99.0.1) and latest Nightly.

On the chat, you've shared a screenshot that shows that your request was blocked by an extension. Remove whatever is blocking the request and it should work. Note that other extensions may be blocking the request (bug 1444729).

The work-around of using window.fetch / window.XMLHttpRequest instead of fetch / XMLHttpRequest seems to work just fine.

You are right.
window.XMLHttpRequest() & window.fetch() are working as expected from page context (& subject to page CORS & restrictions).

The behaviour currently differs from XMLHttpRequest() & fetch() in contentScript context.