Closed Bug 1437748 Opened 7 years ago Closed 5 years ago

webRequest.onBeforeRequest fires two different requestIds after redirect when network cache is filled

Categories

(WebExtensions :: Request Handling, defect, P2)

Product:
WebExtensions
Component:
Request Handling
Version:
58 Branch
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: mozilla+bugzilla, Assigned: MeFisto94)

References

Details

Attachments

(1 file)

testcase-webext.zip
(deleted), application/x-zip-compressed
Details
User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0 Build ID: 20180131010234 Steps to reproduce: Registered webRequest listeners with WebExtension: browser.webRequest.onBeforeRequest.addListener((details) => { console.log('incoming onBeforeRequest', details); }, { urls: ['<all_urls>'], types: ['main_frame'] }, [ 'blocking', 'requestBody' ]); browser.webRequest.onBeforeRedirect.addListener((details) => { console.log('incoming onBeforeRedirect', details); }, { urls: ['<all_urls>'] }, [ 'responseHeaders' ]); browser.webRequest.onCompleted.addListener((details) => { console.log('incoming onCompleted', details); }, { urls: ['<all_urls>'], types: ['main_frame'] }, [ 'responseHeaders' ]); Actual results: Clean profile, no network cache, navigating to http://youtube.com - all requests have the same requestId incoming onBeforeRequest Object { requestId: "5503", url: "http://youtube.com/" .. } incoming onBeforeRedirect Object { requestId: "5503", url: "http://youtube.com/" .. } incoming onBeforeRequest Object { requestId: "5503", url: "https://youtube.com/" .. } incoming onBeforeRedirect Object { requestId: "5503", url: "https://youtube.com/" .. } incoming onBeforeRequest Object { requestId: "5503", url: "https://www.youtube.com/" .. } incoming onCompleted Object { requestId: "5503", url: "https://www.youtube.com/" .. } --- Second time navigating to http://youtube.com with filled cache. After redirecting the requestId changes incoming onBeforeRequest Object { requestId: "5586", url: "http://youtube.com/" .. } incoming onBeforeRequest Object { requestId: "5586", url: "https://youtube.com/" .. } incoming onBeforeRedirect Object { requestId: "5586", url: "https://youtube.com/" .. } incoming onBeforeRequest Object { requestId: "5586", url: "https://www.youtube.com/" .. } incoming onBeforeRequest Object { requestId: "5587", url: "https://www.youtube.com/" .. } incoming onCompleted Object { requestId: "5587" .. } Expected results: The requestId during the second navigation with filled cache should not change. Couldn't find any other information regarding this and since the first navigation doesn't change the requestId and the second navigation doesn't fire onCompleted for the requestId 5586 I'm assuming this is not intended behavior. Also the requestId doesn't change in Chromium under the same circumstances. This popped up while fixing a "two tabs randomly opening"-Issue with Multi-Account Containers (https://github.com/mozilla/multi-account-containers/pull/1120).
Just to add, webRequest.onErrorOccurred doesn't see the request 5586 either - and without being able to correlate the two requests, it's afaik not possible to reliably cancel all requests that originate from the first 5586 request without checking for a combination of tabId and url, which could also affect other requests. In case of Multi-Account Container that would mean canceling clicks on the same link on the originating website. Also tested with Firefox Nightly 60.0a1
Component: Untriaged → WebExtensions: Request Handling
Product: Firefox → Toolkit
Flags: needinfo?(kmaglione+bmo)
Product: Toolkit → WebExtensions
This is probably due to race cache with network, which creates separate requests to the cache service and to the network host. Someone should confirm whether this still happens with RCWN disabled.
Flags: needinfo?(kmaglione+bmo) → needinfo?(mixedpuppy)
Flags: needinfo?(mixedpuppy)
Priority: -- → P2

Tested with Firefox Developer Edition 66.0b10 (64-bit) on Linux and network.http.rcwn.enabled set to false. The issue is still present: after youtube.com redirects to www.youtube.com an additional onBeforeRequests with a new requestId is fired and the original request never fires onCompleted.

Hello,

I can confirm this bug is still present and it is breaking our web extension from working properly, whereas in Chrome it doesn't occur.

From what I have gathered so far, it seems to be a problem with those websites that redirect with 301 status codes rather than 307.
An example of a website with 307 redirection would be : http://www.badoo.com
An example of a website with 301 redirection would be : http://alibaba.ir

Youtube also uses 301 redirection, so it led me to believe that is the cause of the changing RequestId.

Can someone please confirm this? And if yes, can it be classified as a bug? Or is it by design?

Just to add. I tested it with the Disable Cache check-box marked in the Network tab of the Developers Tools.

Another related note: The extension broke even when the cache was empty. I use the web-ext module to debug my extension and it clears the cache every time it is run.
My observation was that onBeforeRequest event was raised with two different RequestIds of which the second one happens after the redirection made by the first request.

Attached file testcase-webext.zip (deleted) —

So according to recent testing this issue doesn't happen with nightly, but does so with all other releases (dev and beta).
Furthermore when using a self-compiled version based on an older commit, which happened before the current nightly was released, the issue is NOT present either.

So it seems there is some different side effect playing into this here. Can someone help testing and/or has a clue why this happens?
The attached zip contains a simple web extension to test it. Go to about:debugging, install it and click on Inspect to open the Console.
Then browse to http://youtube.com (no www no https) a few times and see if the IDs always match up.
Use the main_frame filter in the console to not be spammed.

Note that this web extension will also put a red border around every webpage (borderify example), so that it's really clear whether the web extension is currently active or not.

Okay, using mozregression I was able to find the root of this problem or rather: where it has been fixed as "regression":
"Bug 1456995 (be9a6289)" did fix it and the reason we saw differences between nightly and beta is the presence of the pref dom.serviceWorkers.parent_intercept which is only enabled on nightly currently.

So this is basically solved, the pref will be included as a result of Bug 1588154.

Assignee: nobody → marc.streckfuss
Status: UNCONFIRMED → ASSIGNED
Depends on: 1456995, 1588154
Ever confirmed: true

Closing as this issue is fixed as of Bug 1588154 and should land with Firefox 73.

Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED

Restrict comments here to prevent spam?

Flags: needinfo?(gijskruitbosch+bugs)

Thanks for the heads-up; done.

Flags: needinfo?(gijskruitbosch+bugs)
Flags: needinfo?(aba.alireza)
Restrict Comments: true
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: