Closed Bug 1722217 Opened 3 years ago Closed 1 year ago

Transport-cc constantly and significantly underestimates network bandwidth when PC has 3+ transceivers with transport-cc enabled.

Categories

(Core :: WebRTC, defect, P3)

Firefox 92
defect

Tracking

()

RESOLVED DUPLICATE of bug 1654112

People

(Reporter: yura.yaroshevich, Unassigned)

References

(Depends on 1 open bug, Blocks 1 open bug)

Details

Attachments

(1 file)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:90.0) Gecko/20100101 Firefox/90.0

Steps to reproduce:

An actual scenario I experience issue:

  1. Use Firefox to connect room on SFU server where several other participants send video.
  2. Connection to SFU is done using single peer connection with multiple transceivers to send and receive media from SFU. All transceivers are bundled. Basically I have 2 audio (mic + screen) and 2 video (camera + screen) sendonly transceivers to send audio to SFU and N audio and N video recvonly transceivers to receive media from SFU. All video transceivers have transport-cc enabled, but not audio due to https://bugzilla.mozilla.org/show_bug.cgi?id=1654642.
  3. Observe the quality of video sent to other parties as it received by other parties (assuming SFU has no bugs causing low quality video). Spoiler alert: video quality is bad.

Simple scenario to reproduce issue:
I've also created simple web page which reproduced problem with 2 peer connections running in same browser tab.

  1. https://mstyura.github.io/firefox-webrtc-transport-feedback-bug-repro/ (will reproduce bug, repo: https://github.com/mstyura/firefox-webrtc-transport-feedback-bug-repro/);
  2. https://mstyura.github.io/firefox-webrtc-transport-feedback-bug-repro/?no-transport-cc (will show that without transport cc quality is good, repo: https://github.com/mstyura/firefox-webrtc-transport-feedback-bug-repro/).

Investigation:
The quality of video received by other peers in room is very poor.
That happen because the quality of sent video from sender side is very poor.
The poor video quality on sender side is due wrong bandwidth estimation.
I've tried to debug issue with additional logs and my current high level understanding of the bug is the following.

When RTCP transport feedback message received the copy of it routed to all of the transceivers which has transport-cc enabled: https://searchfox.org/mozilla-central/rev/fffc51a40a341eefced15420d12e15419122b63a/third_party/libwebrtc/webrtc/call/call.cc#1298-1326.
During first handling of transport feedback message the packet sent time is searched in SendTimeHistory and erased from it: https://searchfox.org/mozilla-central/rev/fffc51a40a341eefced15420d12e15419122b63a/third_party/libwebrtc/webrtc/modules/remote_bitrate_estimator/send_time_history.cc#68
Consequent handling of same RTCP transport feedback message by other transceivers will fail to find packet send time in SendTimeHistory and will increase consecutive_delayed_feedbacks_ in DelayBasedBwe https://searchfox.org/mozilla-central/rev/fffc51a40a341eefced15420d12e15419122b63a/third_party/libwebrtc/webrtc/modules/congestion_controller/delay_based_bwe.cc#141-142
It will eventually (after 5 failed attempts to lookup packet sent time) trigger DelayBasedBwe::OnLongFeedbackDelay which will decrease bandwidth estimation.

I also attaching logs captured from modified version of Firefox with extended logs arount RTCP transport feedback processing. Just to confirm what I've just described.

It seems like due to that issue neither https://meet.jit.si/ nor https://meet.google.com/ use transport-cc in Firefox. But https://v3demo.mediasoup.org/ suffer from that issue too (to reproduce it you have to switch between camera and screenshare multiple times, so additional transceivers are created).

Actual results:

The quality of video received by other peers in room is very poor. That happen because the quality of sent video from my side is very poor. The poor video quality of sending side is because wrong bandwidth estimation.

Expected results:

The quality of sent video is high due to proper estimation of network bandwidth.

It seems like Chromium has such issue some time ago as well:

  1. https://bugs.chromium.org/p/webrtc/issues/detail?id=10864
  2. https://bugs.chromium.org/p/chromium/issues/detail?id=1002875

Chromium's WebRTC also has patch with very promising name: https://webrtc-review.googlesource.com/c/src/+/152668

That looks like something worth considering for a cherry-pick, depending on when we're going to next land an update (and if it includes that fix).

Flags: needinfo?(docfaraday)

This looks like it will be included with bug 1654112. Also, bug 1654112 will hopefully unblock bug 1654642 so we can have transport-cc for audio.

Severity: -- → S3
Depends on: 1654112
Flags: needinfo?(docfaraday)
Priority: -- → P3
Depends on: 1725739

Fixed with libwebrtc update.

Status: UNCONFIRMED → RESOLVED
Closed: 1 year ago
Duplicate of bug: 1654112
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: