Closed Bug 1247687 Opened 9 years ago Closed 2 years ago

Implement worker modules

Categories

(Core :: DOM: Workers, task)

task

Tracking

()

RESOLVED FIXED
111 Branch
Tracking Status
firefox111 --- fixed

People

(Reporter: jonco, Assigned: yulia)

References

(Depends on 2 open bugs, Blocks 10 open bugs)

Details

(Keywords: dev-doc-complete, DevAdvocacy)

Attachments

(15 files, 22 obsolete files)

(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
Allow workers to execute module scripts too. Proposed spec: https://github.com/whatwg/html/pull/608
That would be really useful
Keywords: DevAdvocacy

Does this actually need JS engine changes?

Flags: needinfo?(jcoppeard)

No, all the work is in the embedding.

Component: JavaScript Engine → DOM: Workers
Flags: needinfo?(jcoppeard)

Chromium will send an intent to ship ES modules for Shared Workers.
(crbug: https://bugs.chromium.org/p/chromium/issues/detail?id=824646)

We are considering FireFox is positive for supporting this feature since the specification discussion (https://github.com/whatwg/html/pull/608#issuecomment-183003447).
If there is misunderstanding or any updates, please let us know.

As part of overhauling the scriptloader to support modules we'd also like to try and reduce code duplication between the worker scriptloader and the main-thread window scriptloader. Much of the complexity in the worker scriptloader relates to 2 things:

  1. Channels can't be opened on worker threads. Bug 1528285 will address this so we want to wait on that.
  2. Loading ServiceWorker scripts from Cache API storage does not look like using a channel. For this, a tentative plan to investigate is to create a dedicated channel type for loading scripts from the Cache API. This could help normalize things from the script loader's perspective.
    • This may or may not also dovetail with the need for there to be a Worker equivalent to DocumentChannel for ServiceWorker and SharedWorker cases where we need to know the COEP header value before we select the process to spin up the worker.
Depends on: omt-networking

A quick update on this ticket would be appreciated!

The missing JS module support still affects the neo.mjs framework dev mode.
This one now runs fine in Chrome as well as Safari Tech Preview, while FF can only use the dist versions.

I got invited into the working draft podcast this evening to talk about workers.
So far I can only say: "No clue if and when this will happen for FF", which feels bad.

Thx for your input!
Tobias

I am currently working on the dependency (https://bugzilla.mozilla.org/show_bug.cgi?id=1311726), so this work should follow shortly. The prep work is pretty tricky. It will happen, can't give an absolute time estimate right now.

Assignee: nobody → ystartsev
Depends on: 1713615

Hi Yulia, thanks for the fast response! Very much appreciated.

As a dev, I would never ask other devs for absolute time estimates :)
It sounds great, that this ticket is finally getting picked up.

In case you need some rather complex testing scenarios, feel free to use the dev version of the neo online examples:
https://neomjs.github.io/pages/node_modules/neo.mjs/dist/production/apps/website/index.html#mainview=examples&childview=devmode

Best regards,
Tobias

Yulia: thanks for the last update on this! May I ask if "this work should follow shortly" is still accurate?
https://bugzilla.mozilla.org/show_bug.cgi?id=1311726 also references import maps, and I'd understand if that's more important.

I want to encourage/contribute to bundlers to improve their worker support by treating new Worker(new URL("./file.js"), import.meta.url)) similar to a dynamic import in their dependency graph. Library authors like me currently have to work around a lot of issues, but module workers would solve a lot of them.
(I don't want to gripe too much, but I'm speaking from a lot of experience. I've spent literally hundreds of hours finding hacks so that I can provide web worker libraries that are compatible across common JS environments: https://github.com/lgarron/web-worker-compat-problems)

Unfortunately, I think it will hard to get bundlers to do something until Firefox at least experimentally supports module workers, because it's very hard to transpile ESM to something that works in classic workers. It's really valuable if module workers are available in the stable versions of all modern browsers, and Firefox will soon be the only one without support: https://caniuse.com/?search=module%20worker
For example, Snowpack explicitly states:

ESM syntax (import/export) in Web Workers is still not supported in all modern browsers. Snowpack v3.0.0 and the Snowpack Webpack v5 plugin will both support automatic bundling once released.
https://www.snowpack.dev/guides/web-worker/

Is there something that contributors can do to help, if we don't have a lot of knowledge of the Firefox codebase?

I've started work on a contribution to esbuild (for
https://github.com/evanw/esbuild/issues/312) to automatically handle new Worker(new URL("./file.js", import.meta.url)) but this will definitely rely on module worker support.

It seems iOS 15 is shipping in 4 days, at which point Firefox will unfortunately be the only modern browser that requires projects to maintain workarounds for classic modules. 😭

Could I ask again whether we can still hope for Firefox to support module workers in the near future? 🥺

Bug 1311726 seems active, which is currently a blocker of this feature.

Depends on: 1742437, 1742438
Attached file WIP: Bug 1247687 - WIP (obsolete) (deleted) —

Depends on D134052

Blocks: 1350370

Big Ups for this magnificent enhancement. Hope its going well.

Depends on D147313

Depends on D147319

Depends on D147320

Depends on D147322

Depends on D147323

Depends on D147324

Attachment #9255754 - Attachment description: WIP: Bug 1247687 - (WIP) Update WebIDL to accept module workers → WIP: Bug 1247687 - Update WebIDL to accept module workers
Blocks: 1771082
Blocks: 1775574
Attachment #9278186 - Attachment description: WIP: Bug 1247687 - Implement Module Loading for Workers → WIP: Bug 1247687 - Implement Static Module Loading for Workers
Depends on: 1783190
Depends on: 1784457

Comment on attachment 9278178 [details]
WIP: Bug 1247687 - Implement IsTopLevel to handle cases where we have a child module

Revision D147319 was moved to bug 1784457. Setting attachment 9278178 [details] to obsolete.

Attachment #9278178 - Attachment is obsolete: true
Depends on: 1784476
Depends on: 1784477
Depends on: 1784481
Depends on: 1784482

Comment on attachment 9278182 [details]
WIP: Bug 1247687 - Make ModuleLoaderBase thread-insensitive

Revision D147323 was moved to bug 1784477. Setting attachment 9278182 [details] to obsolete.

Attachment #9278182 - Attachment is obsolete: true

Comment on attachment 9278175 [details]
WIP: Bug 1247687 - Move mClientInfo and mController to the WorkerScriptLoader constructor

Revision D147316 was moved to bug 1784476. Setting attachment 9278175 [details] to obsolete.

Attachment #9278175 - Attachment is obsolete: true

Comment on attachment 9278176 [details]
WIP: Bug 1247687 - Make ClientInfo persistant for main scripts on WorkerScriptLoader

Revision D147317 was moved to bug 1784476. Setting attachment 9278176 [details] to obsolete.

Attachment #9278176 - Attachment is obsolete: true

Comment on attachment 9278177 [details]
WIP: Bug 1247687 - Move creation of ScriptLoadRequests into own functions

Revision D147318 was moved to bug 1784482. Setting attachment 9278177 [details] to obsolete.

Attachment #9278177 - Attachment is obsolete: true

Comment on attachment 9278181 [details]
WIP: Bug 1247687 - Move shutdown operation to be always after ProcessRequests

Revision D147322 was moved to bug 1784482. Setting attachment 9278181 [details] to obsolete.

Attachment #9278181 - Attachment is obsolete: true

Comment on attachment 9278184 [details]
WIP: Bug 1247687 - Implement corrected base URI for module redirection

Revision D147325 was moved to bug 1784482. Setting attachment 9278184 [details] to obsolete.

Attachment #9278184 - Attachment is obsolete: true

Comment on attachment 9278180 [details]
WIP: Bug 1247687 - Implement single script loading for main script

Revision D147321 was moved to bug 1784482. Setting attachment 9278180 [details] to obsolete.

Attachment #9278180 - Attachment is obsolete: true

Comment on attachment 9278172 [details]
WIP: Bug 1247687 - Implement ScriptLoaderInterface Skeleton in WorkerScriptLoader

Revision D147313 was moved to bug 1784481. Setting attachment 9278172 [details] to obsolete.

Attachment #9278172 - Attachment is obsolete: true

Comment on attachment 9278173 [details]
WIP: Bug 1247687 - Use ScriptLoaderInterface method for reporting errors

Revision D147314 was moved to bug 1784481. Setting attachment 9278173 [details] to obsolete.

Attachment #9278173 - Attachment is obsolete: true

Comment on attachment 9278174 [details]
WIP: Bug 1247687 - Use ScriptLoaderInterface method for filling compile options

Revision D147315 was moved to bug 1784481. Setting attachment 9278174 [details] to obsolete.

Attachment #9278174 - Attachment is obsolete: true
Attachment #9255755 - Attachment is obsolete: true
Attachment #9255753 - Attachment is obsolete: true
Attachment #9255754 - Attachment description: WIP: Bug 1247687 - Update WebIDL to accept module workers → Bug 1247687 - Update WebIDL to accept module workers; r=asuth
Attachment #9278183 - Attachment description: WIP: Bug 1247687 - Add WorkerModuleLoader skeleton → Bug 1247687 - Add WorkerModuleLoader skeleton; r=jonco
Attachment #9278185 - Attachment description: WIP: Bug 1247687 - Implement InitModuleLoader methods for WorkerScriptLoader and WorkerGlobalScope → Bug 1247687 - Implement InitModuleLoader methods for WorkerScriptLoader and WorkerGlobalScope; r=jonco
Attachment #9278186 - Attachment description: WIP: Bug 1247687 - Implement Static Module Loading for Workers → Bug 1247687 - Implement initial Static Module Loading for Workers; r=jonco

There is a follow on that needs to implement the csp correctly, and also update the wpt tests. I'm
running those now. I will likely flag workers off initially, so that I can land the csp for workers
separately, unless this isn't seen as a major blocker.

Depends on D155566

This is a slightly annoying thing that can happen. When we abruptly cancel (such as an infinitely
running script being forcibly terminated) we will be in a state where the EvaluateModule call will
finish after the loader is destroyed. So, instead we track if there has been a forcible
cancelation, and exit early.

Depends on D155567

Attachment #9278188 - Attachment description: WIP: Bug 1247687 - Disallow ImportScripts from module workers → Bug 1247687 - Disallow ImportScripts from module workers; r=jonco

Depends on D155568

Attachment #9278187 - Attachment is obsolete: true

Depends on D155567

Attachment #9291733 - Attachment is obsolete: true
Attachment #9291729 - Attachment description: WIP: Bug 1247687 - Use referrer to set secFlags for modules → WIP: Bug 1247687 - Use requestCredentials to set secFlags for modules
Attachment #9291730 - Attachment description: WIP: Bug 1247687 - Initial wpt-tests → WIP: Bug 1247687 - Initial wpt-tests; r=jonco,asuth
Attachment #9291731 - Attachment description: WIP: Bug 1247687 - (WIP) Show failing csp tests → Bug 1247687 - enable CSP WPT tests for Static Module Workers; r=evilpie,asuth
Attachment #9291729 - Attachment description: WIP: Bug 1247687 - Use requestCredentials to set secFlags for modules → WIP: Bug 1247687 - Use requestCredentials to set secFlags for modules; r=jonco

Depends on D155692

Attachment #9291730 - Attachment description: WIP: Bug 1247687 - Initial wpt-tests; r=jonco,asuth → Bug 1247687 - Initial wpt-tests; r=jonco,asuth
Attachment #9291729 - Attachment description: WIP: Bug 1247687 - Use requestCredentials to set secFlags for modules; r=jonco → Bug 1247687 - Use requestCredentials to set secFlags for modules; r=jonco
Blocks: 1792824
Blocks: 1792825
Attachment #9292501 - Attachment description: Bug 1247687 - Fix csp for Module Workers; r=evilpie,asuth → Bug 1247687 - Implement csp for Module Workers; r=evilpie,asuth
Severity: normal → S3
Depends on: 1797327
Attachment #9291731 - Attachment is obsolete: true
Blocks: 1798554
Depends on: 1800496
Attachment #9300631 - Attachment is obsolete: true
Attached file WIP: Bug 1247687 - wip service workers (obsolete) (deleted) —

Depends on D156103

Attachment #9305626 - Attachment is obsolete: true

Depends on D156103

Depends on: 1805676
Blocks: 1805676
No longer depends on: 1805676
Pushed by ystartsev@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/85b827971a48 Call EnsureAndLoadStringBundle from Worker ScriptLoaderRunnable; r=allstarschh https://hg.mozilla.org/integration/autoland/rev/34680059b9f0 Update WebIDL to accept module workers; r=asuth https://hg.mozilla.org/integration/autoland/rev/4b0a4fcc6894 Add WorkerModuleLoader skeleton; r=jonco https://hg.mozilla.org/integration/autoland/rev/6c129bd72b61 Implement InitModuleLoader methods for WorkerScriptLoader and WorkerGlobalScope; r=jonco https://hg.mozilla.org/integration/autoland/rev/6ee318df6641 Disallow ImportScripts from module workers; r=jonco https://hg.mozilla.org/integration/autoland/rev/9276c7e1ddd9 Implement initial Static Module Loading for Workers; r=jonco https://hg.mozilla.org/integration/autoland/rev/4dbd510fb042 Implement correct referrer for modules; r=jonco https://hg.mozilla.org/integration/autoland/rev/e7b103c79b1a Implement requestCredentials for Module Workers; r=jonco https://hg.mozilla.org/integration/autoland/rev/0c9650a1ac48 Use requestCredentials to set secFlags for modules; r=jonco https://hg.mozilla.org/integration/autoland/rev/40b8abaf1c0b Handle cancellation of long running modules; r=jonco https://hg.mozilla.org/integration/autoland/rev/354711cf113a Initial wpt-tests; r=jonco,asuth https://hg.mozilla.org/integration/autoland/rev/55f13fb4ee3f Implement csp for Module Workers; r=evilpie,asuth,rpl,ckerschb https://hg.mozilla.org/integration/autoland/rev/2429599729cb Copy over csp related web-platform test files and metadata; r=asuth https://hg.mozilla.org/integration/autoland/rev/5d05ab0c7cde Update wpt code to enable worker static-import templates to set CSP; r=asuth https://hg.mozilla.org/integration/autoland/rev/c6c5750cf713 Update mozilla wpt meta files with new expectations; r=asuth https://hg.mozilla.org/integration/autoland/rev/721f612fd09f 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687: apply code formatting via Lando
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/37501 for changes under testing/web-platform/tests

Comment on attachment 9292502 [details]
WIP: Bug 1247687 - Implement modules for shared workers

Revision D156103 was moved to bug 1805676. Setting attachment 9292502 [details] to obsolete.

Attachment #9292502 - Attachment is obsolete: true

Comment on attachment 9308022 [details]
WIP: Bug 1247687 - Update WPT tests for shared module workers

Revision D164573 was moved to bug 1805676. Setting attachment 9308022 [details] to obsolete.

Attachment #9308022 - Attachment is obsolete: true

Comment on attachment 9304751 [details]
WIP: Bug 1247687 - wip service workers

Revision D162744 was moved to bug 1360870. Setting attachment 9304751 [details] to obsolete.

Attachment #9304751 - Attachment is obsolete: true
Regressions: 1805884

Backed out for frequent string bundle related crashes with PDF viewer (bug 1806064)

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Target Milestone: 110 Branch → ---
Upstream PR was closed without merging
Pushed by ystartsev@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/0ab366c6eaaf Update WebIDL to accept module workers; r=asuth https://hg.mozilla.org/integration/autoland/rev/c94a9dc60dff Add WorkerModuleLoader skeleton; r=jonco https://hg.mozilla.org/integration/autoland/rev/68b476066248 Implement InitModuleLoader methods for WorkerScriptLoader and WorkerGlobalScope; r=jonco https://hg.mozilla.org/integration/autoland/rev/7770ec4717fd Disallow ImportScripts from module workers; r=jonco https://hg.mozilla.org/integration/autoland/rev/0ae1fd421d4f Implement initial Static Module Loading for Workers; r=jonco https://hg.mozilla.org/integration/autoland/rev/49f18430c465 Implement correct referrer for modules; r=jonco https://hg.mozilla.org/integration/autoland/rev/59207e959b7c Implement requestCredentials for Module Workers; r=jonco https://hg.mozilla.org/integration/autoland/rev/45de9ffeec19 Use requestCredentials to set secFlags for modules; r=jonco https://hg.mozilla.org/integration/autoland/rev/9153182c650d Handle cancellation of long running modules; r=jonco https://hg.mozilla.org/integration/autoland/rev/31888ffde37a Initial wpt-tests; r=jonco,asuth https://hg.mozilla.org/integration/autoland/rev/1700d5b79273 Implement csp for Module Workers; r=evilpie,asuth,rpl,ckerschb https://hg.mozilla.org/integration/autoland/rev/0bc871906e97 Copy over csp related web-platform test files and metadata; r=asuth https://hg.mozilla.org/integration/autoland/rev/2adf67f910e8 Update wpt code to enable worker static-import templates to set CSP; r=asuth https://hg.mozilla.org/integration/autoland/rev/d698041e5174 Update mozilla wpt meta files with new expectations; r=asuth https://hg.mozilla.org/integration/autoland/rev/81d052cabf84 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687: apply code formatting via Lando
Regressions: 1806738
Flags: needinfo?(ystartsev)
Upstream PR was closed without merging
Pushed by ystartsev@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/87b6a9494bfb Update WebIDL to accept module workers; r=asuth https://hg.mozilla.org/integration/autoland/rev/ebeb65b42212 Add WorkerModuleLoader skeleton; r=jonco https://hg.mozilla.org/integration/autoland/rev/997e9ca2e1e9 Implement InitModuleLoader methods for WorkerScriptLoader and WorkerGlobalScope; r=jonco https://hg.mozilla.org/integration/autoland/rev/15eb9daca661 Disallow ImportScripts from module workers; r=jonco https://hg.mozilla.org/integration/autoland/rev/474668f6615e Implement initial Static Module Loading for Workers; r=jonco https://hg.mozilla.org/integration/autoland/rev/7327e6438343 Implement correct referrer for modules; r=jonco https://hg.mozilla.org/integration/autoland/rev/f3204b9ec0dc Implement requestCredentials for Module Workers; r=jonco https://hg.mozilla.org/integration/autoland/rev/c585aea29f0c Use requestCredentials to set secFlags for modules; r=jonco https://hg.mozilla.org/integration/autoland/rev/a1900e8b640c Handle cancellation of long running modules; r=jonco https://hg.mozilla.org/integration/autoland/rev/78addb6af24d Initial wpt-tests; r=jonco,asuth https://hg.mozilla.org/integration/autoland/rev/98a751f6f41d Implement csp for Module Workers; r=evilpie,asuth,rpl,ckerschb https://hg.mozilla.org/integration/autoland/rev/9e97be53fb49 Copy over csp related web-platform test files and metadata; r=asuth https://hg.mozilla.org/integration/autoland/rev/060c0649c2f5 Update wpt code to enable worker static-import templates to set CSP; r=asuth https://hg.mozilla.org/integration/autoland/rev/cb4aa6f21ff7 Update mozilla wpt meta files with new expectations; r=asuth https://hg.mozilla.org/integration/autoland/rev/ad56233b442a 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687: apply code formatting via Lando
Attachment #9278179 - Attachment is obsolete: true
Flags: needinfo?(ystartsev)

FF111 MDN docs work for this can be tracked in mdn#24402. Essentially this is a release note and an update to browser compatibility data.

Regressions: 1824498
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: