The highest quality simulcast stream will cut out for approx. 3 seconds with Jitsi.
Categories
(Core :: WebRTC: Audio/Video, defect, P1)
Tracking
()
Tracking | Status | |
---|---|---|
firefox84 | --- | fixed |
People
(Reporter: ng, Assigned: mjf)
References
Details
Attachments
(1 file)
(deleted),
text/x-phabricator-request
|
Details |
The bandwidth estimation causes a dropout of video with Jitsi on the largest of simulcast stream.
Assignee | ||
Comment 1•4 years ago
|
||
Changing the min bitrate from 600k to 200k to allow the HD stream to stay
on more consistently.
Updated•4 years ago
|
Reporter | ||
Comment 2•4 years ago
|
||
The code assumes that the streams will be a constant 30FPS, which isn't true in practice. This should give us more leeway in the minimum bitrate.
Assignee | ||
Comment 3•4 years ago
|
||
As a word of explanation after much digging around:
- As simulcast streams are stopped due to reduced available bandwidth on the send-side,
the receive side will freeze for 3 seconds. This has been verified when Chrome is
the receiver as well. This seems to be a Jitsi bug. - Since we were seeing freezes more often than when Chrome was the sender that could
only mean that we were switching from the HD to SD stream more often than when Chrome
was the sender. This pointed to a codec configuration issue. - We had a very large gap between the LD+SD target bitrates (650k) and the HD minimum
bitrate of 600k. This meant that available bitrate would need to get to 1250k before
we started sending the HD stream. If the bitrate dropped below 1250k, we would stop
sending the HD stream causing the 3 second freeze on the receive side.
The minimum bitrate for the HD stream comes from here:
https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/dom/media/webrtc/libwebrtcglue/VideoStreamFactory.cpp#34
The code for assigning the bitrate allocations for simulcast is here:
https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/simulcast_rate_allocator.cc#37-88
RTCPReceiver::TriggerCallbacksFromRtcpPacket eventually causes VideoSender::SetChannelParameters[1] to execute which stores the new bitrate allocation in encoder_params_
here[2];
[1] https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/third_party/libwebrtc/webrtc/modules/video_coding/video_sender.cc#215
[2] https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/third_party/libwebrtc/webrtc/modules/video_coding/video_sender.cc#234
VideoSender::AddVideoFrame is calling SetEncoderParameters (with encoder_params_
) which eventually calls VP8EncoderImpl::SetRateAllocation.
Whether a stream is transmitted or not is decided here in send_stream_:
https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc#955
The send_stream_ variable is set here in response to VP8EncoderImpl::SetRateAllocation:
https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc#309
On the receiving side, the freeze happens here waiting for a frame:
https://searchfox.org/mozilla-central/rev/d866b96d74ec2a63f09ee418f048d23f4fd379a2/third_party/libwebrtc/webrtc/video/video_receive_stream.cc#420-421
Finally, the key to nailing this issue down was forcing the HD simulcast stream to off,
and then predictably changing between the LD and SD streams approximately every 10 seconds.
This allow the freeze on downgrading streams to be easily observed. The patch to try
this is here:
diff --git a/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc b/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc
--- a/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc
+++ b/third_party/libwebrtc/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc
@@ -299,10 +299,22 @@ int VP8EncoderImpl::SetRateAllocation(co
}
}
+static std::vector<unsigned int> fake_kbps{150, 500, 0};
+static time_t last_change(time(0));
+
size_t stream_idx = encoders_.size() - 1;
for (size_t i = 0; i < encoders_.size(); ++i, --stream_idx) {
+time_t now = time(0);
+if (now - last_change > 10) {
+if (fake_kbps[1]) fake_kbps[1] = 0;
+else fake_kbps[1] = 500;
+printf("MJF: %p VP8EncoderImpl::SetRateAllocation switch to: %s\n",
+ this, (!fake_kbps[1]?"LD":"SD"));
+last_change = now;
+}
unsigned int target_bitrate_kbps =
- bitrate.GetSpatialLayerSum(stream_idx) / 1000;
+ fake_kbps[stream_idx];
+// bitrate.GetSpatialLayerSum(stream_idx) / 1000;
bool send_stream = target_bitrate_kbps > 0;
if (send_stream || encoders_.size() > 1)
Assignee | ||
Comment 4•4 years ago
|
||
Before gtest fix using mach try auto
and adding media mochitests:
https://treeherder.mozilla.org/jobs?repo=try&revision=d197c78ddef5bec8809cbbc8953151a12ab4a38e
After gtest fix:
https://treeherder.mozilla.org/jobs?repo=try&revision=65eacf17b2d0e80ba7f16610435fed4e22e9f834
Pushed by mfroman@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/4ad213bd3283 allow HD simulcast stream to start a bit earlier. r=ng
Comment 6•4 years ago
|
||
bugherder |
Description
•