+++ This bug was initially created as a clone of Bug #1688981 +++

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.80 Safari/537.36

Steps to reproduce:

1. Open one (or both) of the following webRTC test pages in Firefox with Airpods connected: https://webrtc.github.io/test-pages/src/single-audio/ or https://webrtc.github.io/test-pages/src/audio-and-video/
2. Remove your Airpods from your ears and put them back in their case, effectively disconnecting that audio source

Actual results:

1. Video freezes

Expected results:

1. Video should not freeze

When I did a mozRegression on Mac, it failed with "Unable to find enough data to bisect" here:

Tested mozilla-central build: 2020-01-03 (verdict: b)
Tested mozilla-central build: 2020-01-02 (verdict: g)

https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=14ca7276922f986f08f803b28b6d0309f8a081c0&tochange=fc3002cb85dc4d2c44d8aec1246f0356097411e3

Also the detail behaviors are interesting. First the OS:

1. When I take the airpods out of my ears, macOS switches output to speakers, but airpod mic remains hot (tap them together)
2. When I put them back in my ears, macOS switches output back to airpods
3. When I put them in the case, the airPod microphone turns off

In the bad (b) build:

1. Video freezes as soon as I take the airPods out of my years (suggesting this is tied to output?)
2. If I put them back into my ears, the video resumes (!)
3. If I take them out again the video freezes again
4. If I put them in the case, the microphone track ends ("ended" printed on screen)

In the "good" (g) build:

1. Video keeps playing when I take the airPods out of my years, airpod mic remains hot (tap them together)
2. If I put them in the case, the microphone track ends ("ended" printed on screen)

Interestingly, the video freezes and resumes exactly the same way in Safari (14.0.2 and 14.1). Chrome does not freeze. 🤷🏼‍♂️

Turns out this is Automatic Ear Detection 😆

Alastor, is this expected behavior?

Seems confusing for real-time (unbuffered) media like a web conference. Maybe even a potential privacy issue if someone thinks they're muted in this case, when the other party can still see and hear them (verified with https://jsfiddle.net/jib1/r9x5tchq/show in two browser windows).

Should we perhaps limit this to non-realtime (as in buffered) media, like YouTube?

(verified with https://jsfiddle.net/jib1/r9x5tchq/show in two browser windows)

Note it only pauses unmuted video elements, which was a clue. If I turn off auto ear detection they don't pause either.

I'm going to guess media controller here, either bug 1606588 or bug 1571493 from the patchlog.

The only downside I can really think of if we constrain MediaController pausing to only apply to non-MediaStream media providers is that the OS might automatically switch to speakers which start blaring something from an ongoing call or webrtc-based broadcast.

I don't have Airpods , so I'm not able to test that. But could you help me confirm that if the media element was paused by here? If so, we can simply add a constraint to avoid the media with real-time streaming won't be affected by media control.

(In reply to Andreas Pehrson [:pehrsons] from comment #5)

The only downside I can really think of if we constrain MediaController pausing to only apply to non-MediaStream media providers is that the OS might automatically switch to speakers which start blaring something from an ongoing call or webrtc-based broadcast.

Will Airpods automatically get disconnected from computer by Automatic Ear Detection? It sounds like that it just sends a media key singal to pause media, but it still keep connecting, which means the sound would still be sent to Airpods?

(In reply to Alastor Wu [:alwu] from comment #6)

Will Airpods automatically get disconnected from computer by Automatic Ear Detection? It sounds like that it just sends a media key singal to pause media, but it still keep connecting, which means the sound would still be sent to Airpods?

Could very well be. ni? jib to check that and the confirmation.

(In reply to Alastor Wu [:alwu] from comment #6)

I don't have Airpods , so I'm not able to test that. But could you help me confirm that if the media element was paused by here?

Confirmed in the debugger.

If so, we can simply add a constraint to avoid the media with real-time streaming won't be affected by media control.

WFM, but see below.

(In reply to Andreas Pehrson [:pehrsons] from comment #5)

The only downside I can really think of if we constrain MediaController pausing to only apply to non-MediaStream media providers is that the OS might automatically switch to speakers which start blaring something from an ongoing call or webrtc-based broadcast.

This is what happens in Chrome, which appears to have this exception for MediaStreams AFAICT. That is: ear detection still affects Youtube in Chrome, but not the demo links in comment 0 which causes audio redirected to speakers, thanks to macOS.

It's bad for those local-loop demos, and causes microphone feedback.

But demos aside, and for real WebRTC calls, this doesn't seem terrible at all. I mean I clearly can't "pause" my other participants from speaking, and having them muted by this action seems surprising. — I think I would still expect to hear them talking if I take off my AirPods mid-call. That others can hear them now even seems like a feature. I think that's the right action, and is what would happen if I finished putting them in the case anyway, or if I was using and disconnecting any other brand of headsets. If that's undesirable, I can mute the tab.

Will Airpods automatically get disconnected from computer by Automatic Ear Detection? It sounds like that it just sends a media key singal to pause media, but it still keep connecting, which means the sound would still be sent to Airpods?

The AirPods overall remain connected, and the AirPod microphone is still active, but macOS instantly redirects sound output to the MacBook Pro Speakers. E.g. With comment 0 in Chrome, if I remove the AirPods and click them together, I hear clicking sounds coming out of the laptop speakers.

I'm marking this S2 since it represents a regression in behavior from the view of an end-user (i.e. the introduction of a desirable feature extended too far to cover areas where it is not desirable).

[Tracking Requested - why for this release]: Not a major issue, just causing some confusion for AirPod users in web conferencing.

S3 and longstanding issue, doesn't sound like I need to track for 87.

we can simply add a constraint to avoid the media with real-time streaming won't be affected by media control.

Sounds like this is what we need to do (I assume you mean a condition, not a constraint). Where exactly would this go?

Let me know if you need help testing this.

Would it be possible to leave other volume controls alone, by narrowly only defeating the video pausing? This seems like it would achieve parity with Chrome here.

Sorry, I will start working this in next Fx release. But I'm not sure how to determine if the input stream is from real time media or not, (eg. input media stream can be captured from another media element), do we have any way to distinguish them?

(In reply to Alastor Wu [:alwu] from comment #14)
(eg. input media stream can be captured from another media element)

In this case, for all intents and purposes, it's considered real-time.

Okay, thank you!

https://hg.mozilla.org/mozilla-central/rev/3bfc9bc50e19

The patch landed in nightly and beta is affected.
:alwu, is this bug important enough to require an uplift?
If not please set status_beta to wontfix.

For more information, please visit auto_nag documentation.