Open Bug 1316448 Opened 8 years ago Updated 2 years ago

Missing support for RTP CVO extension

Categories

(Core :: WebRTC: Signaling, defect, P4)

49 Branch
defect

Tracking

()

People

(Reporter: anterbg, Unassigned)

References

Details

(Whiteboard: [need info drno 2016-11-11])

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:49.0) Gecko/20100101 Firefox/49.0 Build ID: 20161019084923 Steps to reproduce: 1. WebRTC at iOS receives a call from WebRTC at Firefox 49.0.2 2. WebRTC at iOS answers the call and establishes a video chat with the firefox browser Note: WebRTC at iOS was built as static library from master at commit number# 6ceab083225dfa485bb48f9e43231c216acccbaf Firefox was running Mac OS 10.11 Video feed is properly oriented for a call between iOS and Chrome 54 Some investigations & thoughts Recent WebRTC builds support rtp extension for CVO ( a=extmap:4 urn:3gpp:video-orientation). SDP coming from firefox does not indicate that it supports this extension. I guess if the ff 49 uses a recent WebRTC build, it should offer this extension in its SDP Actual results: When the iPhone device is oriented in portrait, local video feed at ios device and remote video feed at firefox client is rotated at 270 degrees Expected results: When the iPhone device is oriented in portrait, local video feed at ios device and remote video feed at firefox client to be properly rotated
Component: Untriaged → WebRTC: Audio/Video
Product: Firefox → Core
This contains a custom build of WebRTC. Is it something that we can support. Also reporter investigation mentions a problem with SDP. Does it make sense?
Flags: needinfo?(drno)
Whiteboard: [need info drno 2016-11-11]
Component: WebRTC: Audio/Video → WebRTC: Signaling
Firefox 49 (and even all version up to 52) use webrtc.org code from WEBRTC_SVNVERSION 7864 WEBRTC_PULL_DATE "Wed Dec 10 12:23:33 EST 2014" which does not have support for CVO. We are currently working on integrating webrtc.org branch 49. But that will probably land in Firefox 53. And I believe even that has no support for CVO.
Status: UNCONFIRMED → NEW
backlog: --- → webrtc/webaudio+
Rank: 35
Ever confirmed: true
Flags: needinfo?(drno)
Priority: -- → P3
Summary: webrtc video feed coming from ios device is rotated → Missing support for RTP CVO extension
Note also that CVO should not be used if the extension isn't successfully negotiated; if you're using CVO without negotiation (or without checking if it succeeded, you'll get problems like this. When CVO is not negotiated, you should rotate the captured image before encoding (or the webrtc.org code should). Perhaps you're telling it to use CVO always, and so it isn't rotating for you.
Any update on this?, looks like this is missing either stable/dev releases, the problem is that new versions of WebRTC seems to have dropped previous method to handle rotation in favor of CVO, so no matter if the extension is not negotiated Firefox will not be able handle rotation properly.
Looks like our current webrtc.org 49 branch has some support for CVO http://searchfox.org/mozilla-central/search?q=symbol:_ZN6webrtc12RtpExtension14kVideoRotationE&redirect=false Randell: any idea how much effort it would be to activate this? Is it simply signaling it, like we do now with abs send time, or would it require also changes to the encoding and/or rendering side?
Flags: needinfo?(rjesup)
It would require a fair bit of hookup to the code outside of the webrtc.org stack, first to have this info come along from the camera (lots of layers to touch), and then to deal with handling the "should we rotate" on the reception side which involves knowing about the screen/media-element orientation. Signaling it in the webrtc.org code is the easy part
Flags: needinfo?(rjesup)
Randell: Do you know if it is possible somehow to fallback to the previous behavior by not signaling CVO (not enough) and tell WebRTC from clients (say IOS/Android) to send right WxH, HxW for each frame? I've posted a comment on their issue but not answer yet. Thanks.
(In reply to zevarito from comment #7) > Randell: Do you know if it is possible somehow to fallback to the previous > behavior by not signaling CVO (not enough) and tell WebRTC from clients (say > IOS/Android) to send right WxH, HxW for each frame? I've posted a comment on > their issue but not answer yet. Thanks. I can't force them to send something other than not negotiating CVO, which is what we do today (and have always done). It clearly sounds like their bug. We might be able to negotiate CVO but never send it (or never send anything bug "0 degrees") without much more work than the SDP work and some control surfaces in VideoConduit.cpp. Webrtc.org will receive with CVO; the trick will be to ensure when that happens the webrtc.org code does the rotate, or we'll have to do it on the OnFrame()/etc call.
Randell: thanks for the update, I will keep digging on how this can be fixed/workaround from WebRTC side. Please update this issue when CVO gets implemented, Thanks!
Mass change P3->P4 to align with new Mozilla triage process.
Priority: P3 → P4
any update on this topic? I might be wrong but supporting this extension will help mobile client to do less work (especially on vp8 codec which is not very well hardware accelerated optimized) on rotating frame when encoding.

Gentle ping

Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.