audio "autoplay after page interaction" ignores keyboard events
Categories
(Core :: Audio/Video, defect, P2)
Tracking
()
People
(Reporter: michiel, Assigned: alwu)
References
Details
Attachments
(4 files, 1 obsolete file)
|
(deleted),
text/x-phabricator-request
|
lizzard
:
approval-mozilla-beta+
|
Details |
|
(deleted),
text/x-phabricator-request
|
Details | |
|
(deleted),
text/x-phabricator-request
|
Details | |
|
(deleted),
text/x-phabricator-request
|
Details |
Audio elements are not allowed to play on a page that has not been interacted with by the user, and that's fine, but apparently "keyboard interaction" does not count as interaction for this purpose and that's not okay. People who can't use mice should still be able to play web games with sound even though they're navigating purely with a keyboard.
STR:
<html>
<audio id="clip" src="some-random-file.mp3"></audio>
<button id="test">play sound</button>
<script>
let clip = document.getElementById('clip');
let btn = document.getElementById('test');
btn.addEventListener('click', evt => {
clip.play();
});
btn.focus();
</script>
</html>
load page (which will make sure all the user has to do is press enter), and press enter.
Expected:
The clip plays because the user has interacted with the page.
Actual:
Console error "Autoplay is only allowed when approved by the user, the site is activated by the user, or media is muted."
Additional:
This works 100% fine if the user clicks the button using the mouse, so something's forgetting to record the fact that keyboard interaction is just as valid as mouse interaction.
What do other browsers do:
Edge: plays audio on "enter" from keyboard
Chrome: plays audio on "enter" from keyboard
|
||
Updated•6 years ago
|
|
Assignee | |
Updated•6 years ago
|
|
|
Assignee | |
Comment 1•6 years ago
|
||
carriage return and space are common keys which user might use to start media, so we should take account them as vaild user gesture inputs.
tab is also a common interactive key which user will use, and Chrome treats it as a valid input as well.
As their pseudo char code are zero, we have to check their key code in order to distinguish them from other controls keys such as shift, alt...
|
|
Assignee | |
Comment 2•6 years ago
|
||
Except printable keys, we would treat carriage return, space and tab as valid user gesture inputs. Other keys won't activate the document.
|
|
Assignee | |
Comment 3•6 years ago
|
||
We can listen for the block event to know whether AudioContext hasn't started yet which can speed up the test.
|
|
Assignee | |
Comment 4•6 years ago
|
||
Testing web audio with GUM is not related with the original purpose of this test. In order to reduce the complexity of this test, separate this part as another new test.
If I'm reading #2 correctly, that would rule out "space" and "tab", which should definitely count as the user having activated the document. (tab for obvious reasons, space because it acts as the same signal as carriage return for buttons etc)
|
|
Assignee | |
Comment 6•6 years ago
|
||
(In reply to Pomax [:pomax] from comment #5)
If I'm reading #2 correctly, that would rule out "space" and "tab", which should definitely count as the user having activated the document. (tab for obvious reasons, space because it acts as the same signal as carriage return for buttons etc)
What #2 are you referring to? Did you mean the patch1?
It won't rule out the space, it will still treat space as a valid user gesture input. However, before applying my patches, we haven't supported tab.
I tested tab on Chrome, they treat it as a valid input. So maybe we should also allow it in order to reduce the compat issue.
Ah sorry, no: I meant comment #2.
|
||
Updated•6 years ago
|
|
|
Assignee | |
Comment 8•6 years ago
|
||
(In reply to Pomax [:pomax] from comment #7)
sorry, no: comment #2.
Has added tab as a valid input, thank you for reminding.
|
|
Assignee | |
Comment 9•6 years ago
|
||
In order to reduce web compat issue, we should also allow arrow keys and the tab key as valid user gesture input, which Chrome has allowed to active the document.
|
||
Updated•6 years ago
|
|
|
||
Updated•6 years ago
|
|
||
Comment 10•6 years ago
|
||
First off, our intent with gesture activation is to unblock media autoplay when the user has demonstrated intent to autoplay, rather than just when they've interacted with a page.
We don't consider interaction with the browser as demonstration of intent to play media. So pressing keys which cause the browser to scroll the viewport, for example, should not gesture activate the page.
I think space is already gesture activating.
Try this:
- Open https://pearce.org.nz/video/autoplay.html in Firefox with autoplay blocked
- Press space.
- Wait... after a few seconds, a timeout (set on page load) runs and script will call play().
- Observe that video starts to play.
So space is unblocking autoplay today in Firefox. While pressing space does cause the browser to scroll its viewport, it also is commonly used as a keyboard shortcut to play/pause media in desktop media player software. Firefox's built in video controls also use space to play/pause media. So I'm OK with it gesture activating.
If you try the above procedure but press Tab or Enter instead of Space, autoplay does not start. So Tab and Enter don't unblock autoplay, as pointed out in comment 0 above.
I don't think it makes sense to gesture activate on Tab press. Tab is used to shift focus, which doesn't demonstrate intent to autoplay media.
An Enter keypress outside of an editable element isn't interaction with the browser, so I'd be OK with Enter on non-editable controls causing gesture activation.
|
|
||
Updated•6 years ago
|
|
||
Comment 11•6 years ago
|
||
|
||
Comment 12•6 years ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/7c1b053e161b
https://hg.mozilla.org/mozilla-central/rev/f706441a86b5
https://hg.mozilla.org/mozilla-central/rev/9ce43f1e7bc0
https://hg.mozilla.org/mozilla-central/rev/07c49703b70c
|
|
Assignee | |
Comment 13•6 years ago
|
||
Comment on attachment 9046807 [details]
Bug 1530220 - part1 : allow some non-printalble keys as supported user gesture inputs to activate document.
Beta/Release Uplift Approval Request
- Feature/Bug causing the regression: Bug 1452536
- User impact if declined: User won't be able to activate docuement to allow autoplay by pressing
enterkey. - Is this code covered by automated tests?: Yes
- Has the fix been verified in Nightly?: No
- Needs manual test from QE?: Yes
- If yes, steps to reproduce:
- goto https://alastor0325.github.io/htmltests/autoplay_tests/autoplay_test2.html
- press
enterkey - video should start playing
- List of other uplifts needed: all patches in this bug
- Risk to taking this patch: Low
- Why is the change risky/not risky? (and alternatives if risky): These patches doesn't introduce any other behavioural changes, we just extend our support key list in order to let
enterkey activate the document. In addition, we also have an automation test for this issue. - String changes made/needed: none
|
||
Comment 14•6 years ago
|
||
Observed the issue with 66.0b13.
Fix verified with 67.0a1 (2019-03-05) on Windows 10, macOS 10.13.6, Ubuntu 18.04.
Comment on attachment 9046807 [details]
Bug 1530220 - part1 : allow some non-printalble keys as supported user gesture inputs to activate document.
Adds keyboard control for block autoplay, verified in nightly.
OK for uplift to beta 14.
|
||
Comment 16•6 years ago
|
||
| bugherder uplift | ||
|
|
||
Updated•6 years ago
|
|
||
Updated•6 years ago
|
|
Reporter | |
Comment 17•6 years ago
|
||
@cpearce it feels like your argument for why space is supported, but tab isn't, undermines itself: if space is supported because it's commonly used in desktop media players, then there should be no objection to supporting tab as well, as that's commonly used to activate the document in desktop browsers:
- put https://pearce.org.nz/video/autoplay.html in chrome's URL bar and hit enter,
- wait a few seconds,
- tab into the document,
- the video starts playing.
Given that Chrome effectively defines what is "common" these days (as much as I wish that weren't true), that feels reason enough to add support for tab as well, as that's what people will be used to by now.
(I'd mention what Edge does, but given that that's about to become Chrome, its behaviour is going to be "whatever Chrome already does", so that just adds to the argument that tab is commonly used to activate the document).
|
|
||
Comment 18•6 years ago
|
||
Verified with 66.0b14 as well.
|
|
||
Comment 19•6 years ago
|
||
Hrm, doesn't look like this needs developer docs afterall.
Description
•