Closed Bug 1607781 Opened 5 years ago Closed 4 years ago

meet.google.com / Google Hangouts doesn't work with resistFingerprinting ("Couldn't start the video call because of an error")

Categories

(Core :: WebRTC, defect, P2)

Desktop
All
defect

Tracking

()

RESOLVED FIXED
mozilla78
Tracking Status
firefox76 --- wontfix
firefox77 --- fixed
firefox78 --- fixed

People

(Reporter: alex, Assigned: padenot)

References

(Blocks 1 open bug, )

Details

Attachments

(2 files)

+++ This bug was initially created as a clone of Bug #1586423 +++

STR:

  1. (optional) use a fresh Firefox profile
  2. Start a meeting at https://meet.google.com/ , or visit https://meet.google.com/neg-kmjm-uct (a meeting I started; not sure how long it'll be active)

EXPECTED RESULTS:
Underneath the permission prompts, you should see a video preview area and "What's your name?"

ACTUAL RESULTS:
The page instead says "Couldn't start the video call because of an error". (There are still permissions prompts for my video/audio, but I'm still unable to join the meeting even after I accept these prompts with the "remember" checkbox.)

This seems to be a regression from bug 1470568.

I can't reproduce with nightly.

Try using FF72.0, non-Nightly (and ideally on macOS).

I cannot reproduce on 72 release on OS X.

Maybe meet is messing up version detection somehow? It looks like they use the up-to-date syntax for simulcast whenever I use it, but the logging I see here has the old syntax:

https://bugzilla.mozilla.org/show_bug.cgi?id=1586423#c24

I'm also unable to reproduce this using Firefox 72 on OS X. (Or Firefox 74 on OS X or Linux). I was using an @mozilla.com account, I wonder if it's possible we're hitting different versions of meet depending upon the account in use?

Based on Comment 4, it sounds like Byron and I might be getting a different version of Meet than the reporter. :stpeter, is this something you could be able to follow up on with the Meet team? I don't have contacts there and Nils is away. Thank you!

Flags: needinfo?(stpeter)

This might be something wrong with Firefox, but not what I'm reporting.

I originally tested with all my plugins enabled, and it seemed broken.

I then tested with all plugins disabled, and it still seemed broken.

I then tested many times after actually removing all my plugins, one by one, reloading the Meet page after each plugin was removed. Meet was STILL broken.

I then cloned the files making up my profile into a new profile and launched a separate copy of Firefox with that profile (via button to do so in about:profiles), and the problem still persisted.

Finally I force-quit the cloned Firefox, removed the saved session state from the cloned profile, and relaunched it... and it worked.

I have a million (fine, actually several hundred) tabs open across perhaps 20 windows, and every time I've updated Firefox there has been no problem; the saved session state was resumed flawlessly and everything Just Worked. The moment I updated to 72.0, however, Meet was broken. This is why I opened the bug.

How can I troubleshoot this further without clobbering everything in the session?

Hi Alexander, thank you for debugging this further. I wonder if you are getting bad content in your cache. Does it work if you bypass the cache (shift+reload, https://support.mozilla.org/en-US/questions/1073264)?

The cache would have come along for the ride when the rest of the profile was cloned, no?

Either way, Shift+Reload didn't help.

It sounds like this is the case from Comment 7, but can you confirm that this works for you with a fresh profile? Thanks!

Yes, Meet works with a completely new, non-cloned profile. It looks like I have to put a bullet in the old one unless I can troubleshoot it further.

Thanks for reporting and testing. Please ni me again if I need to reach out to the Meet team.

Flags: needinfo?(stpeter)

I've seen a similar issue (also fixed with a fresh profile) using whereby.com. All video elements are frozen.

Dropping this to a P2 since it doesn't affect everyone using meet. It worked for me on an old profile, for instance. I'm not sure if this actually a WebRTC bug or something else.

Priority: P1 → P2

Hello,
Users who are using Kaltura's Mediaspace are reporting a similar behaviour with our product which uses webRTC:
Their complaint:
"When attempting to record a video using the Express Capture in the Firefox v. 71.0 browser, the counter goes to 0 and gets stuck."
We are able to reproduce it on windows machines versions 71 and 72 (not every time, but most of the time), macOS Firefox version is not affected.
Can you please help to fix this issue?

Saeed from the Meet team here.

Alexander, when Meet fails with an error message, there should be a button in the page that lets you file feedback. If you're able to repro this issue again, I'd appreciate if you could file feedback and mention my name in your feedback. I will take a look from our side.

@Saeed — Done. Your name (and a link to your comment here) was included.

(In reply to adir.versano from comment #15)

Hello,
Users who are using Kaltura's Mediaspace are reporting a similar behaviour with our product which uses webRTC:
Their complaint:
"When attempting to record a video using the Express Capture in the Firefox v. 71.0 browser, the counter goes to 0 and gets stuck."
We are able to reproduce it on windows machines versions 71 and 72 (not every time, but most of the time), macOS Firefox version is not affected.
Can you please help to fix this issue?

Please file a separate bug, and cc me on it.

I can reproduce this on FF 72.0.1, Windows 7 64-bit at will. Have filed the feedback with Meet team. Seeing:

OperationError: SIPCC Failed to parse SDP: SDP Parse Error on line 15:  Warning: Unrecognized attribute (x-google-flag) 
SDP Parse Error on line 36:  Warning: Unrecognized attribute (x-google-flag) 
SDP Parse Error on line 60:  Warning: Unrecognized attribute (x-google-flag) 
SDP Parse Error on line 30: No rid attribute for 'rid=f'
SDP Parse Error on line 30: No rid attribute for 'rid=f'
Attached file webrtcDevtool.html (deleted) —

Alex, which version of Firefox were you using? Your user agent string seems to suggest Firefox 68, but the underlying API is behaving like Firefox 71+. Hence the error in negotiating.

Do you have any extensions that might be spoofing your UA string?

@Saeed Yes; I have Facebook Container, HTTPS Everywhere, Privacy Badger, and uBlock Origin installed. It had been version 71 which was working fine, then upon upgrading to version 72.0 Meet became broken. The report was submitted at your request with either 72.0 or 72.0.1.

No progress for a month, and this is still broken in 72.0.2 for me. I have a similar configuration to Alex, but the problem persists with all plugins turned off. Ask me anything.

(In reply to joant from comment #23)

No progress for a month, and this is still broken in 72.0.2 for me. I have a similar configuration to Alex, but the problem persists with all plugins turned off. Ask me anything.

Does this happen with a fresh profile? You have upgraded to 72, but for some reason Meet thinks you're still running an old version of Firefox, and is using an obsolete syntax as a result.

Saeed, Firefox supports the new syntax on all supported versions, are you seeing a lot of usage on versions lower than 68? Maybe it would be best if Meet switched to the new syntax for all Firefox?

Flags: needinfo?(saeed)
Flags: needinfo?(joant)

(In reply to Byron Campen [:bwc] (PTO until Feb 5) from comment #24)

(In reply to joant from comment #23)

No progress for a month, and this is still broken in 72.0.2 for me. I have a similar configuration to Alex, but the problem persists with all plugins turned off. Ask me anything.

Does this happen with a fresh profile? You have upgraded to 72, but for some reason Meet thinks you're still running an old version of Firefox, and is using an obsolete syntax as a result.

Found it. privacy.resistFingerprinting was true, which reports the FF version as the ESR version (68.0). Toggling this flag allows access to Meet. The flag also seems to disable overriding the userAgent via add-ons or the general.useragent.overridepreference setting.

I'll revert to canvas fingerprinting blocking via an add-on for now.

Saeed, Firefox supports the new syntax on all supported versions, are you seeing a lot of usage on versions lower than 68? Maybe it would be best if Meet switched to the new syntax for all Firefox?

That'd fix this problem for sure.

So I wasn't crazy after all! Like joant, I also have privacy.resistFingerprinting enabled; this is why the problem remained with all plugins deleted but was resolved with a fresh profile.

I had been using Safari for Meet as a workaround in the interim, but now that the root cause seems to have been identified, I'll wait for a fix. This goes to show how troublesome server-side filtering for user-agent can be!

(In reply to Alex Burke from comment #26)

This goes to show how troublesome server-side filtering for user-agent can be!

Hear hear!

Status: UNCONFIRMED → RESOLVED
Closed: 5 years ago
Resolution: --- → INVALID
Flags: needinfo?(saeed)
Flags: needinfo?(joant)

Can I ask why this was closed as INVALID?
Comment #25 and Comment #26 both indicate the issue that is occurring here, with the privacy.resistFingerprinting config setting.
Yes this is the result of Google doing server-side filtering for a user-agent but is there nothing Mozilla can/will do to resolve this?

Flags: needinfo?(docfaraday)

(In reply to Ben Montour from comment #30)

Can I ask why this was closed as INVALID?
Comment #25 and Comment #26 both indicate the issue that is occurring here, with the privacy.resistFingerprinting config setting.
Yes this is the result of Google doing server-side filtering for a user-agent but is there nothing Mozilla can/will do to resolve this?

This is Google's bug, yes.

Flags: needinfo?(docfaraday)

Have we already notified Google about this?

(If not, perhaps better to morph this into a WebCompat bug, and reopen it to track Google addressing this on their end? I'm sure the WebCompat folks would be happy to craft an email to Google folks who may be able to file a bug on their end.)

Flags: needinfo?(docfaraday)

Saeed works on Meet, yes. We might be able to escalate via WebCompat.

Flags: needinfo?(docfaraday)

(In reply to Daniel Holbert [:dholbert] from comment #32)

Have we already notified Google about this?

(If not, perhaps better to morph this into a WebCompat bug, and reopen it to track Google addressing this on their end? I'm sure the WebCompat folks would be happy to craft an email to Google folks who may be able to file a bug on their end.)

Thank you! I appreciate it being brought up with the WebCompat team for some visibility into a fix.

Saeed, do you know if this issue is tracked and/or being worked on somewhere on the Google/Meet end?

Summarizing the situation here: it seems that Google Meet is taking different codepaths for Firefox users, based specifically on the reported Firefox version from the UA string -- but that's not great, because the UA string isn't meant to be a feature-detection surface, and the privacy.resistFingerprinting about:config pref causes the reported version number to be frozen at 68 even in newer Firefox versions. And apparently Meet's "Firefox 68" codepath doesn't work with the WebRTC version in current Firefox versions.

Is it possible for Meet to do feature-detection here, so that things work for people with recent Firefox/WebRTC and with privacy.resistFingerprinting set to true?

Blocks: meet
Component: WebRTC → Desktop
Flags: needinfo?(saeed)
Product: Core → Web Compatibility
Version: 72 Branch → unspecified
Status: RESOLVED → REOPENED
Ever confirmed: true
Resolution: INVALID → ---

Yes. I am tracking it on our side.

What is Firefox's recommended way of feature-detecting this change? The old SDP parser allowed:
a=simulcast: send rid=foo;bar

While, the new one (I think post Firefox 68), only allowed:
a=simulcast:send foo;bar

We can feature detect by using the old format and seeing if it throws an error. But there are other reasons where applying SDPs might throw an error and so it's probably not great to feature-detect that way.

At this point, I'm considering just using the new format regardless of Firefox version, as the usage of older versions of Firefox that required older SDP is probably very low.


On a side note, re: using UA-based version detection vs feature-detection.
We never want to rely on UA-based detection. But very often that's impossible, specially for issues affecting the media stack. Take a recent example, where Firefox 66 and below reported RTT in milliseconds, whereas Firefox 67+ reported them in seconds per standard.

Flags: needinfo?(saeed)

(In reply to Saeed Jahed from comment #36)

Yes. I am tracking it on our side.

What is Firefox's recommended way of feature-detecting this change? The old SDP parser allowed:
a=simulcast: send rid=foo;bar

While, the new one (I think post Firefox 68), only allowed:
a=simulcast:send foo;bar

We can feature detect by using the old format and seeing if it throws an error. But there are other reasons where applying SDPs might throw an error and so it's probably not great to feature-detect that way.

There are a few ways you could do this.

You could have two simple SDPs that are identical except for the simulcast format; if the new one works, then use the new format. If the new one doesn't, try the old one. If that works, then use the old format. If the old one fails too, that probably means you're on a newer Firefox; either sRD is just broken for some crazy reason, and this is likely newer Firefox because that's the baseline, or you have a newer Firefox that is implementing some new validity checking that your SDP is failing, so you should use the new format. It might also be because you're using Firefox that doesn't support simulcast at all, which means using the new format won't be any worse than using the old.

If all else fails, you could inspect our o= line, which has had the version in it for a really long time. That could be used to reliably detect old Firefox.

At this point, I'm considering just using the new format regardless of Firefox version, as the usage of older versions of Firefox that required older SDP is probably very low.

This would probably be fine too.

If all else fails, you could inspect our o= line, which has had the version in it for a really long time. That could be used to reliably detect old Firefox.

Yes. That's true. But that really does get around the user's intention of trying to hide their Firefox version from us. I'd also imagine that because of that at some point Firefox would probably stop leaking version number there.

You could have two simple SDPs that are identical except for the simulcast format; if the new one works, then use the new format. If the new one doesn't, try the old one.

I think this too complex of a feature-detection strategy for API users to do, and also very error-prone.

What might help, going forward, is to have overlaps between supported formats, and logging deprecation warnings. So in newer versions of FF support parsing both the old and new formats, but log a deprecation notice in the console when old one is used. Then after a few versions pull parsing of the old format.

(In reply to Saeed Jahed from comment #38)

If all else fails, you could inspect our o= line, which has had the version in it for a really long time. That could be used to reliably detect old Firefox.

Yes. That's true. But that really does get around the user's intention of trying to hide their Firefox version from us. I'd also imagine that because of that at some point Firefox would probably stop leaking version number there.

You could have two simple SDPs that are identical except for the simulcast format; if the new one works, then use the new format. If the new one doesn't, try the old one.

I think this too complex of a feature-detection strategy for API users to do, and also very error-prone.

What might help, going forward, is to have overlaps between supported formats, and logging deprecation warnings. So in newer versions of FF support parsing both the old and new formats, but log a deprecation notice in the console when old one is used. Then after a few versions pull parsing of the old format.

We did have such an overlap. Firefox gained support for the new format back in 68, and parsed both until version 72. Additionally, when support for the old format went away (in 72), all supported versions of Firefox supported the new format. Any Firefox that still only accepts the old format is waaaaaaaay out of date by now. So, if it were me implementing this stuff, I would just use the new format unconditionally, and if it did not work, I would urge the user to update their browser!

Agreed, it seems like there are two easy fix options here -- either:
(A) Adjust the UA sniffing logic to check for version 67 or lower (and offer the old version in that case). This may be the most minimal adjustment that would help here. Though note that these Firefox versions are no longer supported anyway.[1]
...or:
(B) Just use the new format unconditionally.

[1] The oldest supported Firefox version is 68 ESR. (The previous ESR, version 60, stopped receiving updates & being supported in October, per the calendar/version-number image near the bottom of https://support.mozilla.org/en-US/kb/firefox-esr-release-cycle )

Thanks Byron and Daniel. The overlap wasn't clear to us in the past. That's really helpful. We'll update on our side.

This should be fixed in Meet now. It'd be great if someone with the issue can verify that it works for them now.

I will run some tests tomorrow and report back. Thank you so much for the follow up!

@Saheed I can confirm that it did not work two days ago, and that it works today. Thanks so much for fixing this up!

Thanks all!

Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → FIXED

So this is improved for me, but still not working as intended. When I enable resistFingerprinting I get into a Meet now, other attendee's can hear me, but I cannot hear them. As soon as I turn resistFingerprinting back off and reload the Meet, audio work again.

[:miketaylr] Ben Montour is right, this is not fixed. resistFingerprinting still seems to block Meet from seeing the microphone, even though the user is prompted for permission to access the mic and FF confirms in the dropdown that mic access is allowed.

The settings window in Meet shows no microphone device available.

Tested on FF 74.0 Win x64.

I guess you can tell that I just tested to make sure the error didn't appear, and didn't actually go through a call with someone else (I was testing the scope of the bug as filed).

@Saeed can you please test this internally with FF74 and privacy.resistFingerprinting set to true and get to the bottom of it?

Status: RESOLVED → REOPENED
Resolution: FIXED → ---

(In reply to joant from comment #47)

[:miketaylr] Ben Montour is right, this is not fixed. resistFingerprinting still seems to block Meet from seeing the microphone, even though the user is prompted for permission to access the mic and FF confirms in the dropdown that mic access is allowed.

The settings window in Meet shows no microphone device available.

Tested on FF 74.0 Win x64.

I had this issue as well. I did get it to finally accept my microphone by removing and re-allowing the permission to do so and restarting the Meet. I could still not hear anybody else on the Meet though. It seems there are a number of pieces that are still not behaving very well with this.

Updated to FF75 (MacOS 10.15.4), behavior persists. Happy to provide any additional context, but not sure how/what.

@Saeed I removed and re-added mic+camera permissions and retested. Created Meet ID ing-vzmy-jvo with myself on both ends (one end was FF76.0 on macOS 10.15.4 19E287 [the second and current build of 10.15.4] with resistFingerprinting enabled, and the other end was Meet 41.0.308913298 on iOS 13.4.1 17E262. In other words, both ends were absolutely up-to-date.

  • Video seemed to work fine in both directions.
  • Outbound audio from Firefox Mac works, but the "microphone activity" widget does not react to microphone audio input.
  • Inbound audio from iOS does not work.

Firefox 76.0.1 with resistFingerprinting: true still has an issue with inbound audio (outbound works just fine) on https://meet.google.com (identical to the symptoms reported by Alex Burke) and causes problems to be reported on https://test.webrtc.org. The latter shows no mic problems with resistFingerprinting: false, but the following occurs when it's true:

On page load, there is a modal gum-dialog element that says:

Welcome to WebRTC Troubleshooter
Failed to access your computer's camera and microphone (NotAllowedError: The request is not allowed by the user agent or the platform in the current context.).

There is a link button to proceed anyway: "Continue without audio/or video". When you choose that to dismiss the modal and click Start, the Microphone > Audio Capture check reports:

[ OK ] Audio track created using device=Internal Microphone
[ FAILED ] Failed to get access to local media due to error: NotSupportedError: AudioContext.createMediaStreamSource: Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported.

This is the same as the dev console warning that occurs at meet.google.com (twice when first joining a meeting you created, and once more when another user joins your meeting):

Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported.

Google support chat insists that this is a Firefox bug, not a Google Meet bug, and their only recommendation is "use Chrome." Can we confirm whether this is a bug with resistFingerprinting or an implementation issue in Google Meet? Is there any information from the console, about:webrtc, or about:config that I can give that would help determine this?

Other voice and video meeting solutions seem to work fine in Firefox, and don't have this particular audio issue. Using Chrome for meetings or disabling resistFingerprinting and all its other benefits (e.g. forced-UTC time zone, which is valued for more than its privacy implications) are certainly options, but my current workaround is to call into every Google Meet session with a mobile phone to get audio, which is less than ideal.

Any ideas about the above comment?

Flags: needinfo?(padenot)
Flags: needinfo?(jib)

Can we confirm whether this is a bug with resistFingerprinting or an implementation issue in Google Meet?

Enabling rFP changes the UA string, and many Google sites rely on the UA string to send what it thinks the browser will support. You could verify this by sending whatever UA string rFP sends in a normal Firefox profile (with rFP unset) and see if it breaks in the same way.

(In reply to Ben Burhans from comment #52)

Firefox 76.0.1 with resistFingerprinting: true still has an issue with inbound audio

Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported.

Thanks Ben, I'm able to reproduce this with https://test.webrtc.org - The web console warning appears here:

  createAudioBuffer: function() {
→   this.audioSource = this.audioContext.createMediaStreamSource(this.stream);

It appears turning on resistFingerprinting breaks all basic web audio with microphone as source (!) E.g. https://jsfiddle.net/jib1/0s769z0d/

I believe resistFingerprinting changes the default sample-rate reported, so that might be a clue. Paul may know more.

Flags: needinfo?(jib)
Summary: meet.google.com / Google Hangouts doesn't work in Firefox 72.0 ("Couldn't start the video call because of an error") → meet.google.com / Google Hangouts doesn't work with resistFingerprinting ("Couldn't start the video call because of an error")
Assignee: nobody → padenot

clearing NI

Flags: needinfo?(padenot)
Pushed by padenot@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/bbe2b442eeb2 Consistently use a fixed sample-rate when resisting fingerprinting. r=jib

@pascalc I see this won't land in 76. That's OK, but will this get backported into ESR 68? Or must we wait until the next ESR? Sorry to pester, asking for an enterprise that's on ESR and struggling with this.

Status: REOPENED → RESOLVED
Closed: 5 years ago4 years ago
Resolution: --- → FIXED

Comment on attachment 9150116 [details]
Bug 1607781 - Consistently use a fixed sample-rate when resisting fingerprinting. r?jib

Beta/Release Uplift Approval Request

  • User impact if declined: Users that have privacy.resistFingerprinting enabled AND a non-44100Hz rate can't use VC service (and some other service that make use of the microphone and Web Audio API)
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: Yes
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): What we're doing is something that can be done with a pref, the code path is well exercised, although there is no test for this particular privacy.resistFingerprinting bit of behaviour.
  • String changes made/needed: none
Attachment #9150116 - Flags: approval-mozilla-beta?

Comment on attachment 9150116 [details]
Bug 1607781 - Consistently use a fixed sample-rate when resisting fingerprinting. r?jib

Fix for Google Meet in combination with the privacy.resistFingerprinting pref, the patch is small and the risk seems low, taking it in our last beta since it affects a top videoconferencing site, thanks.

Attachment #9150116 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

this fix was released? I'm trying to use a different sample rate (16000) in version 78, and the problem continue ..

I disable the fingerprinting and it does not solve as well ..

Any tip? thx

Component: Desktop → WebRTC
OS: macOS → All
Product: Web Compatibility → Core
Hardware: x86_64 → Desktop
Target Milestone: --- → mozilla78
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: