If a user previously clicked "Don't Allow" when Firefox tried to use the camera/mic, when they visit a WebRTC site such as talky.io, the video/camera functionality doesn't work and there isn't any indication why. This is due to new functionality in Mojave that triggers OS-level dialogs letting the user allow/deny access to the camera per-application. See bug 1470833 for more explanation. See screenshot for talky.io example where talky.io is being used after the user clicked "Don't Allow" when presented with the OS camera/mic prompts. In this scenario, we might be able to manually trigger the OS dialog[1] or at least detect that we don't have access and trigger our own dialog instructing the user to enable camera/mic access for Firefox in the macOS "Security and Privacy" settings.

(In reply to Haik Aftandilian [:haik] from comment #0) > Created attachment 8995584 [details] > Behavior of WebRTC site after user has clicked "Don't Allow" on OS > Camera/Mic dialog > > If a user previously clicked "Don't Allow" when Firefox tried to use the > camera/mic, when they visit a WebRTC site such as talky.io, the video/camera > functionality doesn't work and there isn't any indication why. This is due > to new functionality in Mojave that triggers OS-level dialogs letting the > user allow/deny access to the camera per-application. See bug 1470833 for > more explanation. Thanks for bringing this up. Wow... that is probably going to cause some pain. > See screenshot for talky.io example where talky.io is being used after the > user clicked "Don't Allow" when presented with the OS camera/mic prompts. > > In this scenario, we might be able to manually trigger the OS dialog[1] or > at least detect that we don't have access and trigger our own dialog > instructing the user to enable camera/mic access for Firefox in the macOS > "Security and Privacy" settings. Your screen shot shows that Firefox believes that there are no camera devices available. But from your comment it sounds like we can figure out that Firefox hasn't been granted access to cameras. So yes we should in that case bring up some dialog explaining that the web page requests camera access, but the user first needs to give Firefox access privileges in OSX. My only concern would be: even if Firefox hasn't been given access to cameras can we figure out if the system does have any cameras available at all? Because I think it would be confusing to users if Firefox would ask them to allow access to cameras, only so that after access has been granted Firefox tells them "oh you don't even have cameras on this system". Haik can you take care of these additional interactions with OSX?

(In reply to Nils Ohlmeier [:drno] from comment #1) > Your screen shot shows that Firefox believes that there are no camera > devices available. But from your comment it sounds like we can figure out > that Firefox hasn't been granted access to cameras. So yes we should in that > case bring up some dialog explaining that the web page requests camera > access, but the user first needs to give Firefox access privileges in OSX. Sorry, I wasn't clear about that. The screenshot was taken after I'd already clicked "Allow" on Firefox's own per-site doorhanger permission after visiting the site a second time. I clicked "Don't Allow" on the OS dialog the first time visiting the site. Given that I did see our doorhanger with the camera name, I think we are still detecting the camera OK. > My only concern would be: even if Firefox hasn't been given access to > cameras can we figure out if the system does have any cameras available at > all? > > Because I think it would be confusing to users if Firefox would ask them to > allow access to cameras, only so that after access has been granted Firefox > tells them "oh you don't even have cameras on this system". Apple had a presentation at WWDC[1] and the presenter stated that we can enumerate the devices without user consent (at time 14:50 in the video). If that works as described, I think we'll be OK with respect to detecting the camera/mic. > Haik can you take care of these additional interactions with OSX? I'll confirm with my manager. 1. https://developer.apple.com/videos/play/wwdc2018/702/

For precedence, this is how Fennec works: If I go to Settings/Apps/Firefox/Permissions and clear cam and mic, and then: 1. Go to https://jsfiddle.net/jib1/r60bzmrs/ on Fennec then I'm first prompted by the OS with Allow Firefox to record video? ☐ Don't ask again [Deny] [Allow] If I hit "Deny", the webpage gets NotFoundError. If I hit "Allow" I gets the gUM prompt. Then in the gUM prompt, if I hit "Don't share", the webpage gets NotAllowedError if I hit "Share", the webpage gets my camera (there is no in-prompt camera selector on Fennec). Frankly, this wasn't exactly the order and errors I expected. The gUM prompt has more context and seems better to go first. Maybe android limitations dictate this order?

Back to Mojave, since it allows enumeration without authorization, and authorization appears global per type (camera/mic), we could perhaps do better: 1. First check OS for denied/restricted authorization before gum prompt and throw NotAllowedError or NotFoundError respectively? 2. Then open gUM prompt as the first prompt the user sees. 3. If user selects "Don't Allow" in gUM prompt, throw NotAllowedError as usual. 4. Only if user chooses a camera (this is desktop) and hits "Allow" would we invoke the OS dialog. 5. If user for some reason changed their mind from 1 second ago and now hits "Deny" in OS dialog, throw NotAllowedError. 6. webpage gets camera. Pros: the user has more context to base their decision on this way. Cons: The extra prompt may feel more ridiculous at the end, but only once.

Looking at the video in [1] around 15:52 the API for checking if we have access returns {notDetermined, restricted, denied, authorized}. I think we should do the following: 1. JS call gUM() 2. Check if OS is Mojave/new APIs are available 2.1 Enumerate camera devices 2.1.1 If no cameras exist return NotFoundError 2.2 Invoke the API to check if we have access to the camera already 2.3 Depending on the return value: 2.4.1 'notDetermined': invoke the API to request access to the camera with a string like "Firefox wants access to your camera, because the current open page requested access to your camera" 2.4.2 'restricted' or 'denied': return NotAllowedError 2.4.3 'authorized': continue 3. Show the Fx gUM door hanger prompt 4. Proceed as usual... To me it's more logical to first grant access to the Firefox application as a whole from, before Firefox then asks the more detailed question about which camera etc. Especially as OSX doesn't grant on a per device access, but for all cameras or nothing. From watching the video it also appears what Apple recommend to do - probably that is what people are used to already from mobile devices. [1] https://developer.apple.com/videos/play/wwdc2018/702/

Unfortunately, we probably can't fix this until we move to the 10.14 SDK. The API to query the allow/denied status[1] and to trigger the dialog[2] are both listed as 10.14+ on Apple's docs and I don't expect them to be backported to earlier SDK's which means fixing this is blocked by bug 1475652. 1. authorizationStatusForMediaType https://developer.apple.com/documentation/avfoundation/avcapturedevice/1624613-authorizationstatusformediatype 2. requestAccessForMediaType https://developer.apple.com/documentation/avfoundation/avcapturedevice/1624584-requestaccessformediatype?language=objc

(In reply to Haik Aftandilian [:haik] from comment #6) > Unfortunately, we probably can't fix this until we move to the 10.14 SDK. > The API to query the allow/denied status[1] and to trigger the dialog[2] are > both listed as 10.14+ on Apple's docs and I don't expect them to be > backported to earlier SDK's which means fixing this is blocked by bug > 1475652. > > 1. authorizationStatusForMediaType > > https://developer.apple.com/documentation/avfoundation/avcapturedevice/ > 1624613-authorizationstatusformediatype > > 2. requestAccessForMediaType > > https://developer.apple.com/documentation/avfoundation/avcapturedevice/ > 1624584-requestaccessformediatype?language=objc This isn't entirely correct. We are frequently using new APIs that aren't available to us in the SDK we're building with. For TouchBar support (bug 1313429) we're even weak linking to entire classes (rather than just new methods on existing classes, as is the case here). You can have a look at nsCocoaWindow.mm in the following patch to see how to build with new properties, methods etc.: https://bugzilla.mozilla.org/page.cgi?id=splinter.html&ignore=&bug=1236512&attachment=8874376 At runtime, use `respondsToSelector:` to check whether or not the class will respond to the new method.

(In reply to Stephen A Pohl [:spohl] from comment #7) > This isn't entirely correct. We are frequently using new APIs that aren't > available to us in the SDK we're building with. For TouchBar support (bug > 1313429) we're even weak linking to entire classes (rather than just new > methods on existing classes, as is the case here). You can have a look at > nsCocoaWindow.mm in the following patch to see how to build with new > properties, methods etc.: > > https://bugzilla.mozilla.org/page.cgi?id=splinter. > html&ignore=&bug=1236512&attachment=8874376 > > At runtime, use `respondsToSelector:` to check whether or not the class will > respond to the new method. Thanks, I'll give this a try. Removing dependency on bug 1475652 for now.

(In reply to Nils Ohlmeier [:drno] from comment #5) > To me it's more logical to first grant access to the Firefox application as > a whole from, before Firefox then asks the more detailed question about > which camera etc. I checked, and this is also what Firefox for iOS does. Since trust granted to Firefox extends beyond the current page, I agree it feels logical to ask for Firefox first.

I have not been able to get the authorizationStatusForMediaType API to work on 10.14 with my 10.13 SDK build. respondsToSelector or instancesRespondToSelector both return NO for the authorizationStatusForMediaType checks below when run on 10.13 or 10.14. if ([AVCaptureDevice instancesRespondToSelector:@selector(authorizationStatusForMediaType:)] || [captureDevice respondsToSelector:@selector(authorizationStatusForMediaType:)]) { printf("authorizationStatusForMediaType is available\n"); } else { printf("authorizationStatusForMediaType is not available\n"); } Additionally I tried using @available: if (@available(macOS 10.14, *)) { printf("Running on 10.14\n"); AVAuthorizationStatus authStatus = [captureDevice authorizationStatusForMediaType:AVMediaTypeVideo]; printf("authStatus: %u\n", authStatus); } else { printf("Not running on 10.14\n"); } I had to modify AVCaptureDevice.h and comment out the enum AVAuthorizationStatus and the method authorizationStatusForMediaType because those are in the 10.13 SDK, but flagged as only available on iOS and that prevented the compile from succeeding. With @available, the 10.13 build crashes on 10.14 with "unrecognized selector". firefox[1361:58356] -[AVCaptureDALDevice authorizationStatusForMediaType:]: unrecognized selector sent to instance 0x12b9f0380 ** Unknown exception behavior: -2147483647 ** Unknown exception behavior: -2147483647 ** Unknown exception behavior: -2147483647

Additionally, we should add some type of popup notification when the user has granted access to the camera in Firefox, but Firefox doesn't have the OS permission. If we show the OS permission dialog first, this could still happen if the user granted access in Firefox to a site before upgrading to Mojave. If we show the Firefox permission dialog first, we can easily run into this situation. The popup would tell the user the camera/mic can't be accessed due to OS permissions and direct the user to fix it in macOS settings. If possible, have a button that triggers opening the preferences.

I think getUserMedia would simply fail with NotFoundError in that case. That way, web sites can distinguish it from NotAllowedError. This would also cover the case where parental controls may limit access.

(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #12) > I think getUserMedia would simply fail with NotFoundError in that case. That > way, web sites can distinguish it from NotAllowedError. This would also > cover the case where parental controls may limit access. Just to be clear, we would then want sites to change their code to perform the following type of check if (error==NotFoundError && OS==Mojave) { showMessage("Either you don't have a camera or you need to give $BROWSER access in System Preferences->Security & Privacy settings"); }

With this WIP patch, when visiting a WebRTC site, before the Firefox camera/mic prompt is displayed, we call platform code in nsCocoaUtils.mm. For now, the code iterates over the camera or microphone devices, does some respondsToSelector checks, and then attempts to call requestAccessForMediaType when running on 10.14. On 10.14, the call to requestAccessForMediaType() triggers the crash.

Thanks to :spohl for helping to get this working. In the earlier patch, I was using respondsToSelector incorrectly. This updated WIP patch compiles on 10.11 and is able to use the new 10.14-specific API's when executing on 10.14. The code is not complete, but feedback appreciated on the design. The macOS API to request access via the prompt for the camera or microphone is asynchronous. Given that, I'm introducing an IDL |nsIOSPermissionRequest| implemented in platform code to be called from webrtcUI.jsm. Platform code will return a promise that resolves/rejects when the permission is granted/blocked. Specifically the webrtcUI.jsm prompt()[1] function would invoke this code and wait for the result. 1. https://searchfox.org/mozilla-central/rev/3fa761ade83ed0b8ab463acb057c2cf0b104689e/browser/modules/webrtcUI.jsm#343

I filed bug 1487204 to cover the platform work needed for this bug. Once that has landed, this bug can be used for the front end changes required to webrtcUI.jsm.

@jib, do you know of someone that could take over the front-end side of this work. i.e., the changes to webrtcUI.jsm:prompt() to call the new API's I'm adding in bug 1487204? Regarding comment 5 above, I think it makes more sense to show the gUM prompt first and then the Mojave permission prompt. For the case where the user visits a page requesting the camera which they do not want to allow, if the Mojave prompt is shown first, the user might think they have to reject the request to prevent the site from getting access to the camera. If we show the Mojave prompt after the gUM prompt, we'll only show the Mojave prompt to the user for a site they've already allowed. @drno, what do you think?

Johann, are you on Mojave?

(In reply to Haik Aftandilian [:haik] from comment #18) > For the case where the user visits a page requesting the camera which they > do not want to allow, if the Mojave prompt is shown first, the user might > think they have to reject the request to prevent the site from getting > access to the camera. That's a great point. Are there limits to how often we can trigger the Mojave prompt? If there are, then we should definitely not show the Mojave prompt first. If there aren't, I suppose we still have a choice.

(In reply to Jan-Ivar Bruaroey [:jib] (needinfo? me) from comment #20) > (In reply to Haik Aftandilian [:haik] from comment #18) > > For the case where the user visits a page requesting the camera which they > > do not want to allow, if the Mojave prompt is shown first, the user might > > think they have to reject the request to prevent the site from getting > > access to the camera. > > That's a great point. Are there limits to how often we can trigger the > Mojave prompt? > > If there are, then we should definitely not show the Mojave prompt first. Yes, it can just be shown once. Once the user clicks "Don't Allow", we can't manually get it to be shown again, but we can detect that we don't have access.

When the user has allowed the site to access the camera, but the OS prevents Firefox from accessing the camera, we could return "NotReadableError"[1] or possibly "SecurityError"[1]. If those don't fit, we could try to add a new error to the spec for this scenario. NotReadableError Although the user granted permission to use the matching devices, a hardware error occurred at the operating system, browser, or Web page level which prevented access to the device. SecurityError User media support is disabled on the Document on which getUserMedia() was called. The mechanism by which user media support is enabled and disabled is left up to the individual user agent. 1. https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia

NotReadableError is for device trouble, e.g. exclusive OS access conflict with another program. E.g. prompt user to close other programs, unplug/replug and trying again. SecurityError is an access violation error from cross origins or Feature Policy, a programming error. NotFoundError seems best. Either the user just prevented Firefox from ever accessing the devices, or this child lacks parental permission to use the devices. In either case, we want to avoid web sites treating users as if they're withholding IMHO.

This patch adds OS-level permission checks for the camera and mic after the user has granted access within Firefox. On Mojave, if the user hasn't yet granted or denied access to Firefox, the checks trigger an async permission request. If permission is denied at the OS level, but granted to the site within Firefox, return NotFoundError. These changes depend on and are based on the patches posted for review on bug 1487204. I haven't touched the permissions code before and do little front end work so these changes need a detailed/close review.

(In reply to Haik Aftandilian [:haik] from comment #24) > Created attachment 9005538 [details] [diff] [review] > Patch that adds OS-level permission checks for the camera and mic https://treeherder.mozilla.org/#/jobs?repo=try&revision=1b67e9013fc884c25c8b8629b42e8dc9d077aee9 Patch was tested locally on Mojave using jib's https://jsfiddle.net/jib1/r60bzmrs/ As an example, on talky.io, with the fix, when Firefox doesn't have access to the camera, the site displays "Oh no! We can't access your camera/microphone. Unknown error occurred" Without the fix, the site displays the video as black pixels and doesn't report an error. The way the permission API is implemented (tentatively) in platform code (bug 1487204), if it is called from any OS other than 10.14+, it returns status indicating access is available. If a similar camera/mic access API were to be added to Windows, it would be easy to add a Windows-specific implementation.

Haik, once you have it landed in nightly, could you request an uplift to 63 beta? Thanks

I don't have Mojave yet but I just got a new computer and I can make one of them run Mojave. Haik, is this ready for review on the front-end side of things? If so, feel free to flag me for review and I can take a look this week, whenever my Mojave installation is finished :)

(In reply to Johann Hofmann [:johannh] from comment #27) > Haik, is this ready for review on the front-end side of things? If so, feel > free to flag me for review and I can take a look this week, whenever my > Mojave installation is finished :) OK, thanks. I think it's ready for review or feedback if the approach is wrong. (In reply to Pascal Chevrel:pascalc from comment #26) > Haik, once you have it landed in nightly, could you request an uplift to 63 > beta? Thanks Yes, will do.

Check if we have permission from the OS to access the camera and microphone after the user has granted access to a site. If permission is denied at the OS level, but granted to the site within Firefox, return NotFoundError.

Comment on attachment 9005538 [details] [diff] [review] Patch that adds OS-level permission checks for the camera and mic Moved review to Phabricator. Marked this attachment as obsolete.

(In reply to Haik Aftandilian [:haik] from comment #18) > Regarding comment 5 above, I think it makes more sense to show the gUM > prompt first and then the Mojave permission prompt. For the case where the > user visits a page requesting the camera which they do not want to allow, if > the Mojave prompt is shown first, the user might think they have to reject > the request to prevent the site from getting access to the camera. If we > show the Mojave prompt after the gUM prompt, we'll only show the Mojave > prompt to the user for a site they've already allowed. @drno, what do you > think? That is a very valid concern. A totally maybe somewhat crazy alternative could be that if find out that we have never asked the user for camera access that we only do the OS level prompt and skip the Fx prompt, to avoid the confusion caused by double prompts. If we have access already and the page doesn't have permanent permission we would still show the Fx prompt only. The big disadvantage of showing on the initial prompt only the OS level is that the user no longer could choose which camera and mic to be used. And it is a somewhat inconsistent behavior, but that is the case with two vs one prompt as well.

(In reply to Nils Ohlmeier [:drno] from comment #31) > A totally maybe somewhat crazy alternative could be that if find out that we > have never asked the user for camera access that we only do the OS level > prompt and skip the Fx prompt, to avoid the confusion caused by double > prompts. If we have access already and the page doesn't have permanent > permission we would still show the Fx prompt only. The big disadvantage of > showing on the initial prompt only the OS level is that the user no longer > could choose which camera and mic to be used. And it is a somewhat > inconsistent behavior, but that is the case with two vs one prompt as well. I would favor maintaining consistency and always showing the Fx prompt allowing the user to pick the device. Which camera is used could be important for privacy and then there are scenarios where the default camera is wrong. For example, the user has a laptop with the lid closed and they're using an external camera. The user should only ever see the OS prompt once (because the setting is persistent) and if it's after the Fx prompt then there should be enough context for the user to understand why it is needed. And the OS dialog has a different appearance compared to the Fx prompt which may distinguish it as an OS thing.

I’m having some email problems, so I’m reposting my email to Haik below: ----- OS permission dialogue ====================== Haik wrote: > The user should only ever see the OS prompt once (because the setting is persistent) This is still a good idea. We just need to make sure that the System dialogue says something helpful. Maybe something like this? > “Firefox” would like to access the camera. > > You can use Firefox to control websites that are allowed to use and blocked from using the camera. This clarifies that the OS control is different from the browser control. ----- Firefox permission dialogue =========================== Ok. Now let’s attack the problem on our side. The problem is: if you say no to the OS dialogue, then Firefox permissions will fail silently, and the website will appear broken. This scenario can happen on two conditions: * The site has been given access to the camera, but Firefox doesn’t have access (as you wrote) * The site asks for access to the camera, but Firefox doesn’t have acces First of all, Firefox shouldn’t be silent. We should flash a red crossed-out camera icon (https://design.firefox.com/icons/icons/desktop/video-recorder-16-disabled.svg) on the URL bar. And secondly, we should do as Haik wrote in bug 1470833: > if we are ever on a site that has been given access to the camera, but Firefox doesn't have access, we should show a UI mentioning this with a button that takes the user to the relevant System Preferences pane. So, the camera icon would flash, and you click on it. The permission dialogue can say something like this: > Firefox does not have access to the camera > > [Allow access] This clarifies that the user needs to grant Firefox access first, before the website can ask for access (or activate the camera, if it’s already given access). “Allow access” is the button that will take user to System Preferences > Security & Privacy > Privacy > Camera. ----- What do you think of these proposals? Would it help if there’s a mockup for these, or do you think that they’re self-explanatory? If they sound good, I’ll send a request to one of our content strategists (Michelle Heubusch, Brian Jones, Meridel Walkington or Betsy Mikel) so they can think of the right string to use and the right thing to say. Until that point, we have something reasonably understandable to use. And when the new string comes, we can simply replace mine.

(In reply to Bram Pitoyo [:bram] from comment #33) > What do you think of these proposals? Would it help if there’s a mockup for > these, or do you think that they’re self-explanatory? They sound good/appropriate to me. I think they will help avoid cases where users think the camera isn't working due to Firefox not working. I think this bug should still be fixed with the patch posted, because it results in the site showing an error message when Firefox hasn't been granted access to the camera in the OS. And we want sites to detect the error. Sites could even detect the user is on Mojave and provide their own inline help for the user. I can file a bug for these proposals, but the WebRTC team/experts will have to prioritize and take on the work. In the meantime, with the fix for this bug, we won't be silently failing because the site will get an error. > If they sound good, I’ll send a request to one of our content strategists > (Michelle Heubusch, Brian Jones, Meridel Walkington or Betsy Mikel) so they > can think of the right string to use and the right thing to say. We may not be able to work on this right away, but I think having the text would help. > Until that point, we have something reasonably understandable to use. And > when the new string comes, we can simply replace mine. I think we could go with a simple string for now such as "Only sites you allow within Firefox will be able to use the camera." My instinct is that anything longer than 1 sentence requires too much reading and consideration.

Haik, I’ve added the user flow diagram that contains the desired behaviour. Hope this helps!

String proposal: Firefox in-content permission dialogue > Heading: Firefox does not have access to the camera > > Button: [Allow access] <-- will pop open macOS system permission dialogue macOS system permission dialogue > Heading: “Firefox” would like to access the camera. > > Body: Only sites you allow within Firefox will be able to use the camera.

Thanks, Bram. I've filed bug 1494172 to address the additional UI work described in comment 33. This bug is not dependent on that and provides an improvement on its own.

Comment on attachment 9007871 [details] Bug 1479051 - [macOS 10.14] WebRTC sites silently fail if user previously clicked "Don't Allow" for Firefox camera/mic access Johann Hofmann [:johannh] has approved the revision.

Pushed by haftandilian@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/a79effc68816 [macOS 10.14] WebRTC sites silently fail if user previously clicked "Don't Allow" for Firefox camera/mic access r=johannh

Reminder about the uplift to beta. Too late for 62.

Comment on attachment 9007871 [details] Bug 1479051 - [macOS 10.14] WebRTC sites silently fail if user previously clicked "Don't Allow" for Firefox camera/mic access [Beta/Release Uplift Approval Request] Feature/Bug causing the regression: N/A new Mojave issue User impact if declined: User may get into a situation where they have allowed cam/mic permission in Firefox for a site, but Firefox does not have access to the cam/mic and as a result those devices do not work. From the site's perspective, the cam/mic appear to work, but the received camera stream is just black pixels. Is this code covered by automated tests?: No Has the fix been verified in Nightly?: Yes Needs manual test from QE?: Yes If yes, steps to reproduce: On Mojave, use a WebRTC site such such as talky.io and verify that when access to camera/mic is allowed within Firefox, but denied within the OS, the page still displays an error. List of other uplifts needed: Bug 1487204, Bug 1470833 Risk to taking this patch: Medium Why is the change risky/not risky? (and alternatives if risky): The changes are to front end JS code and not complex. The changes use a new API added in dependent bug for Mojave which are also medium risk. The changes are partially covered by automated tests. String changes made/needed: None

This bug depends on bug 1497408 in Nightly. It is only a dependency for uplift if bug 1445943 (which introduced 1497408) is also being uplifted.

Comment on attachment 9007871 [details] Bug 1479051 - [macOS 10.14] WebRTC sites silently fail if user previously clicked "Don't Allow" for Firefox camera/mic access Given that it just landed, and needs 2 other patches uplifted, one of them (bug 1487204) being marked as wontfix for 63 for being platform work too risky to be uplifted in late betas, I am not going to take the uplift as the risk/benefit ratio seems too high for our last beta before RC. Given that this is a tracked 63 bug, if we have a dot release in a few weeks, we will evaluate it as a potential ride along. Thanks Haik for your work on this!