Transport-cc constantly and significantly underestimates network bandwidth when PC has 3+ transceivers with transport-cc enabled.
Categories
(Core :: WebRTC, defect, P3)
Tracking
()
People
(Reporter: yura.yaroshevich, Unassigned)
References
(Depends on 1 open bug, Blocks 1 open bug)
Details
Attachments
(1 file)
(deleted),
text/x-log
|
Details |
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:90.0) Gecko/20100101 Firefox/90.0
Steps to reproduce:
An actual scenario I experience issue:
- Use Firefox to connect room on SFU server where several other participants send video.
- Connection to SFU is done using single peer connection with multiple transceivers to send and receive media from SFU. All transceivers are bundled. Basically I have 2 audio (mic + screen) and 2 video (camera + screen) sendonly transceivers to send audio to SFU and N audio and N video recvonly transceivers to receive media from SFU. All video transceivers have transport-cc enabled, but not audio due to https://bugzilla.mozilla.org/show_bug.cgi?id=1654642.
- Observe the quality of video sent to other parties as it received by other parties (assuming SFU has no bugs causing low quality video). Spoiler alert: video quality is bad.
Simple scenario to reproduce issue:
I've also created simple web page which reproduced problem with 2 peer connections running in same browser tab.
- https://mstyura.github.io/firefox-webrtc-transport-feedback-bug-repro/ (will reproduce bug, repo: https://github.com/mstyura/firefox-webrtc-transport-feedback-bug-repro/);
- https://mstyura.github.io/firefox-webrtc-transport-feedback-bug-repro/?no-transport-cc (will show that without transport cc quality is good, repo: https://github.com/mstyura/firefox-webrtc-transport-feedback-bug-repro/).
Investigation:
The quality of video received by other peers in room is very poor.
That happen because the quality of sent video from sender side is very poor.
The poor video quality on sender side is due wrong bandwidth estimation.
I've tried to debug issue with additional logs and my current high level understanding of the bug is the following.
When RTCP transport feedback message received the copy of it routed to all of the transceivers which has transport-cc enabled: https://searchfox.org/mozilla-central/rev/fffc51a40a341eefced15420d12e15419122b63a/third_party/libwebrtc/webrtc/call/call.cc#1298-1326.
During first handling of transport feedback message the packet sent time is searched in SendTimeHistory and erased from it: https://searchfox.org/mozilla-central/rev/fffc51a40a341eefced15420d12e15419122b63a/third_party/libwebrtc/webrtc/modules/remote_bitrate_estimator/send_time_history.cc#68
Consequent handling of same RTCP transport feedback message by other transceivers will fail to find packet send time in SendTimeHistory and will increase consecutive_delayed_feedbacks_
in DelayBasedBwe
https://searchfox.org/mozilla-central/rev/fffc51a40a341eefced15420d12e15419122b63a/third_party/libwebrtc/webrtc/modules/congestion_controller/delay_based_bwe.cc#141-142
It will eventually (after 5 failed attempts to lookup packet sent time) trigger DelayBasedBwe::OnLongFeedbackDelay
which will decrease bandwidth estimation.
I also attaching logs captured from modified version of Firefox with extended logs arount RTCP transport feedback processing. Just to confirm what I've just described.
It seems like due to that issue neither https://meet.jit.si/ nor https://meet.google.com/ use transport-cc in Firefox. But https://v3demo.mediasoup.org/ suffer from that issue too (to reproduce it you have to switch between camera and screenshare multiple times, so additional transceivers are created).
Actual results:
The quality of video received by other peers in room is very poor. That happen because the quality of sent video from my side is very poor. The poor video quality of sending side is because wrong bandwidth estimation.
Expected results:
The quality of sent video is high due to proper estimation of network bandwidth.
Reporter | ||
Comment 1•3 years ago
|
||
It seems like Chromium has such issue some time ago as well:
- https://bugs.chromium.org/p/webrtc/issues/detail?id=10864
- https://bugs.chromium.org/p/chromium/issues/detail?id=1002875
Chromium's WebRTC also has patch with very promising name: https://webrtc-review.googlesource.com/c/src/+/152668
Comment 2•3 years ago
|
||
That looks like something worth considering for a cherry-pick, depending on when we're going to next land an update (and if it includes that fix).
Comment 3•3 years ago
|
||
This looks like it will be included with bug 1654112. Also, bug 1654112 will hopefully unblock bug 1654642 so we can have transport-cc for audio.
Comment 4•1 year ago
|
||
Fixed with libwebrtc update.
Description
•