[meta] OffscreenCanvas
Categories
(Core :: Graphics: CanvasWebGL, enhancement, P3)
Tracking
()
People
(Reporter: jujjyl, Assigned: aosmond)
References
(Depends on 15 open bugs, Blocks 1 open bug)
Details
(Keywords: meta, parity-chrome, Whiteboard: [gfx-noted])
Reporter | ||
Updated•7 years ago
|
Reporter | ||
Comment 1•7 years ago
|
||
Updated•7 years ago
|
Updated•7 years ago
|
Updated•7 years ago
|
Updated•7 years ago
|
Updated•5 years ago
|
Comment 2•5 years ago
|
||
This shows up as a popular feature on the Web in crawls for Chrome APIs that Firefox doesn't implement.
Comment 3•5 years ago
|
||
(In reply to Jukka Jylänki from comment #0)
Bug 709490 tracked initial implementation of OffscreenCanvas, after which it
is still behind a pref. We don't seem to have a metabug tracking the
remaining work to be able to ship unpreffed, so here is one.
I'd like to point out that user gesture requirements equivalent to Chrome's "User Activation v2" are important for some OffscreenCanvas use cases. In our case we can host our entire game engine in a web worker, forwarding input events over postMessage, and posting back for certain DOM APIs not available in workers (e.g. requestFullscreen). If user activation is limited to synchronous callbacks in input events, this blocks the use of any input-restricted APIs (like requestFullscreen), which would block our use of OffscreenCanvas. With User Activation v2, the restrictions are relaxed enough that it keeps working and means we can use OffscreenCanvas. I'm not sure what the status of user gestures are in Firefox but it might be worth reviewing this for OffscreenCanvas.
Comment 5•5 years ago
|
||
(In reply to ashley from comment #4)
I'd like to point out that user gesture requirements equivalent to Chrome's "User Activation v2" are important for some OffscreenCanvas use cases.
How did you determine that?
Which specification describes user gesture requirements for OffscreenCanvas
?
In our case we can host our entire game engine in a web worker, forwarding input events over postMessage, and posting back for certain DOM APIs not available in workers (e.g. requestFullscreen).
Used a similar approach at https://plnkr.co/edit/4Tb91b?p=info to workaround Firefox, Nightly having not implemented OffscreenCanvasRenderingContext2D
for the purpose of manipulating pixels in a Worker
("However there are currently no reliable way to render a video on an OffscreenCanvas." https://discourse.wicg.io/t/proposal-offscreenvideo/3952).
The image drawn onto <canvas>
using drawImage()
when "2d"
context is set or transferFromImageBitmap()
when ImageBitmapRenderingContext
is not limited by user activation, the issue is the drawn image is not visible (tranparent; width
and height
might not be set at the <canvas>
) on the <canvas>
, see https://bugzilla.mozilla.org/show_bug.cgi?id=1564784#c5.
If user activation is limited to synchronous callbacks in input events, this blocks the use of any input-restricted APIs (like requestFullscreen), which would block our use of OffscreenCanvas. With User Activation v2, the restrictions are relaxed enough that it keeps working and means we can use OffscreenCanvas. I'm not sure what the status of user gestures are in Firefox but it might be worth reviewing this for OffscreenCanvas.
Chromium implementation of user activation is WIP and AFAICT not based on interoperability with any other implementation of an API; e.g., https://bugs.chromium.org/p/chromium/issues/detail?id=1014171.
Not sure why the image is not displayed when OffscreenCanvas
or when ImageBitmapRenderingContext
are used, even though canvas.toDataURL()
outputs a data URL
of the image (.toBlob()
does not).
Comment 6•5 years ago
|
||
(In reply to ashley from comment #4)
I'd like to point out that user gesture requirements equivalent to Chrome's "User Activation v2" are important for some OffscreenCanvas use cases.
This code does not require user activation https://plnkr.co/edit/0By56O?p=preview. Not certain how user activation criteria can be applied to an OffscreenCanvas
within a Worker
, particularly at Firefox, Nightly.
The observable bug with the code in the above-linked plnkr is that transferFromImageBitmap()
"does not work": width
and height
of <canvas>
on main thread does not change, image is not visible on the <canvas>
Main thread
<canvas style="border:1px solid purple"></canvas>
<script>
const canvas = document.querySelector("canvas");
const offscreen = canvas.transferControlToOffscreen();
worker.postMessage({offscreen}, [offscreen]);
</script>
Worker
const bitmap = await createImageBitmap(request);
const {width, height} = bitmap;
const osc = e.data.offscreen;
osc.width = width;
osc.height = height;
const osctx = osc.getContext("bitmaprenderer");
// the image at main thread is visible at Chromium after 2 requestAnimationFrame calls
// the image is not visible at Firefox 70 or Nightly 72
osctx.transferFromImageBitmap(bitmap);
I think you've misunderstood. Of course user gestures are not part of the OffscreenCanvas spec. It is a different spec (at least, I think it is specified), but it is relevant to OffscreenCanvas use cases, which is why I highlighted it.
Comment 8•5 years ago
|
||
(In reply to ashley from comment #7)
I think you've misunderstood. Of course user gestures are not part of the OffscreenCanvas spec. It is a different spec (at least, I think it is specified), but it is relevant to OffscreenCanvas use cases, which is why I highlighted it.
If not mistaken gathered the gist of the comment as to not neglect consideration of user activation relevant to OffscreenCanavas
, correct?
From perspective here, given that the concept of user activation is arbitrary as to the implementation authors decisions, where interoparability might not be expressed as the primary aim of the discrete implemenation umbrella or specific API, was attempting to convey that given the various observable bugs in Firefox implementation of OffscreenCanvas
, if user activation is not an issue now in Firefox, is it beneficial to include such considerations contemporary to observable bugs?
Do bugs currently exist relevant to OffscreenCanvas
and user activation at Firefox?
Comment 10•5 years ago
|
||
(In reply to ashley from comment #9)
I just wanted to make a heads up to the Firefox developers.
Yes, gather that. User activation relevant to Native File System implementation at Chrome reveals at least several cases that could produce unexpected results. Not certain if any single user activation policy could be implemented in the wild to manage all possible use (and edge) cases and implementer interests for any API.
BTW excellent effort on Imagedata extensions. Cheers.
Updated•5 years ago
|
Updated•5 years ago
|
Comment 16•5 years ago
|
||
Is there a timeline of release for OffScreenCanvas support in FF?
Comment 17•5 years ago
|
||
Roughly 2020H2.
Comment 18•4 years ago
|
||
we have an offscreencanvas crash issue windows FF.
Description: OffscreenCanvas crashes on Windows
Steps to Reproduce:
Running below url on Windows 10 with 78.0.2 (64-bit)
https://web.autocad.com/?webgl2=true
set gfx.offscreencanvas.enabled = true in firefox about:config
Open https://web.autocad.com/?webgl2=true then Login/Create a new account
Open any drawing in file manager
Firefox crashes after the page has run for a couple of seconds.
Chrome is working without any problem.
Crash report:
https://crash-stats.mozilla.org/report/index/330920ed-0b09-4e51-b59b-63ca70200720
https://crash-stats.mozilla.org/report/index/9b8524f1-5300-44b8-a6ca-de5e70200713
Comment 19•4 years ago
|
||
Moved the crash signature to a bug on its own, Bug 1657375 to allow tracking it better.
Comment 20•4 years ago
|
||
Any updates on when this is planned to be implemented? If it's still planned to be implemented in 2020H2 that would be awesome.
Comment 21•4 years ago
|
||
Related to bug 1645136
Comment 22•3 years ago
|
||
Any progress? Here Firefox plans to ship WebCodecs API only in workers context
https://github.com/w3c/webcodecs/issues/211#issuecomment-859965775
But you can't draw decoded image without OffscreenCanvas in worker.
Last time I checked it year ago behind the flag on Ubuntu it crashed.
Comment 23•3 years ago
|
||
Can I get an update on prioritization for this work?
Offscreen Canvas does still seem to be super crashy, so I understand why it's flagged off, but this is a key technology for a number of ML workloads, and it's a gap that is very hard to fill with other technologies that Gecko supports.
Comment 24•3 years ago
|
||
(In reply to Richard Newman [:rnewman] from comment #23)
Can I get an update on prioritization for this work?
Offscreen Canvas does still seem to be super crashy, so I understand why it's flagged off, but this is a key technology for a number of ML workloads, and it's a gap that is very hard to fill with other technologies that Gecko supports.
"Ideally this year"
Updated•3 years ago
|
Comment 26•3 years ago
|
||
Note: I am not Jeff Gilbert. I am a different entity with a similar name and interest in OffscreenCanvas (small universe).
Cross linking OffscreenCanvas support in webkit to monitor progress there: https://bugs.webkit.org/show_bug.cgi?id=183720
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Comment 27•3 years ago
|
||
I just want to bring awareness that OffscreenCanvas is required to bring compatibility to a number of WebExtension use cases due to the ongoing switch to forcing service workers instead of background pages. Bug 1573659 and Bug 1578286
Assignee | ||
Comment 28•3 years ago
|
||
(In reply to sdaniele3 from comment #27)
I just want to bring awareness that OffscreenCanvas is required to bring compatibility to a number of WebExtension use cases due to the ongoing switch to forcing service workers instead of background pages. Bug 1573659 and Bug 1578286
Oh good to know, thanks! Currently we have turned it on in nightly for certain domains as an origin trial. The major gap remaining is text rendering with Canvas2D. We intend to close that this year and hopefully it can get turned on for extensions as well :).
Updated•2 years ago
|
Updated•2 years ago
|
Comment 30•2 years ago
|
||
This officially shipped last fall. Misc cleanup bugs and some compliance work is all that's left. Tasks specific to interop should be tracked directly under that meta bug so they are easy to find.
Description
•