User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36 Steps to reproduce: Windows 10 Firefox Nightly Set Flags: media.mediasource.enabled true media.mediasource.format-reader true media.mediasource.webm.enabled true media.format-reader.webm true 1.Watch High frame rate video https://www.youtube.com/watch?v=pF8yeXn9R4Y 2.Select Quality 1080p60 (which will be VP9) Actual results: Video doesn't play continuously. Often video freezes audio continue to play. Expected results: Smooth video playback.

VP9 drops more frames than H264 on my Mac.

H264 would be HW accelerated on your mac; but software only for VP9... not that surprising

(In reply to Jean-Yves Avenard [:jya] from comment #2) > H264 would be HW accelerated on your mac; but software only for VP9... > > not that surprising Let me clarify. VP9 drops more frames than either software or hardware H.264 on Mac.

As I explained to Anthony on IRC ; you will find that the time measured between InputExhausted() and the next Input() is largely irrelevant. The MediaFormatReader (MFR) attempts to have at any given time to have at least two decoded frames in its queue, ready to be returned to the MediaDecoderStateMachine (MDSM) when requested (via RequestVideoData or RequestAudioData). As such, it will feed the MediaDataDecoder (MDD) data via Input() regardless of the decoder calling InputExhausted() or not. Unlike the H264 decoder that will typically requires between 2 and 16 frames before outputting one ; the VPXDecoder usually decode a frame immediately, so the size of the decoder's input queue is rarely > 1. Not long after MDD::Input() is called MFR::Output is called back. As the MFR will call VPXDecoder::Input() consecutively, regardless of InputExhausted(), the MDD's TaskQueue is never empty: When VPXDecoder::Input gets called; there will typically be another one pending waiting for the previous one to complete. If the decoder's TaskQueue isn't empty, the InputExhausted() is never fired by the decoder. When it will be fired, it will be because there are no more input on its way, and that's because the MFR has already more than 2 decoded samples queued ; it doesn't need new ones anymore : the decoder can go idle. Measuring the time between InputExhausted() and Input() for this particular decoder, really only measures the time the MediaFormatReader has decided that the decoder can be idle for. More relevant to speed of decoding, looking at FFmpeg's use of the libvpx decoder: https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/libvpxdec.c#L52 or chrome: https://code.google.com/p/chromium/codesearch#chromium/src/remoting/codec/video_decoder_vpx.cc&q=vpx_codec_dec_init&sq=package:chromium&l=149 You can see that they set the number of threads to use when calling vpx_codec_dec_init. We don't: http://mxr.mozilla.org/mozilla-central/source/dom/media/platforms/agnostic/VPXDecoder.cpp#57 we pass nullptr as config . my guess is libvpx doesn't use multi-threaded decoding and that's why the performance in Firefox is so poor.

libvpx with nullptr as config: time to play 10s of 4K: 49.58s libvpx with config set, 2 threads: 38s. Still not fast enough ; but a 24% improvement nevertheless.

I'm investigating, so will take this bug.

(In reply to Dan Glastonbury :djg :kamidphish from comment #9) > Created attachment 8673467 [details] [diff] [review] > Schedule State Machine when VideoQueue() is low on frames. When video is under produced: If UpdateRenderedVideoFrames() can't find a frame in VideoQueue() that is `in the future`, ie. has a time stamp that is greater than clock time, the thread is scheduled in 40ms. The logic in UpdateRenderedVideoFrames() always leaves the current frame sent to the compositor at the head of the queue. It's time stamp is always less than clock time. Because there's always one, old frame in the queue, the logic in OnVideoDecoded() needs to reschedule the state machine when queue size is two, after adding the new frame.

web-platform-tests: https://treeherder.mozilla.org/#/jobs?repo=try&revision=e3de9d3e9e02

Comment on attachment 8673467 [details] [diff] [review] Schedule State Machine when VideoQueue() is low on frames. Approval Request Comment [Feature/regressing bug #]: 1193614 [User impact if declined]: Video playback will show jank. [Describe test coverage new/current, TreeHerder]: current media tests [Risks and why]: This low-risk patch helps to improve playback when video decode is underproducing. It schedules the the frame to be displayed on decode, which should have been happening. [String/UUID change made/needed]: None

Comment on attachment 8673467 [details] [diff] [review] Schedule State Machine when VideoQueue() is low on frames. Improve youtube, taking it. Should be in 42 beta 8.

I'm hitting a conflict trying to uplift this to beta because bug 1195632 hasn't been uplifted. Could you post a branch-specific patch for this?

Chris, can you help here? Thanks

Patch for beta.

Dan, your fix has landed on central, aurora, and beta. Is there any more work needed before we can remove close this bug (and remove the leave-open keyword)?

unfortunately, bug 1213131 (which is the first patch attached to this ticket) uplift was denied, so there's not going to be any benefit on beta now. The core issue that makes 60fps impossible with VP9 on mac isn't fixed by either of those fixes.

Patches on this bug don't completely fix the issue. Do we have a meta bug for playback at 60fps?

(In reply to Dan Glastonbury :djg :kamidphish from comment #25) > Patches on this bug don't completely fix the issue. Do we have a meta bug > for playback at 60fps? No, so I guess this bug is as good as any. :)

sorry had to back this out for android m9 perma failures like https://treeherder.mozilla.org/logviewer.html#?job_id=16725447&repo=mozilla-inbound starting with this push

Doubt it has anything to do with it. Are we talking about the same bug? Which backout?

I think you should backout bug 1218311.

As Jean-Yves says, but I'll take a look at my patch.

https://hg.mozilla.org/integration/mozilla-inbound/rev/d84d28b5c9be The sheriff has backed out bug 1218311. Bug 1193614 is fine here.

I've stopped working on this for now.

A user posted about this long back - https://www.reddit.com/r/firefox/comments/433eer/this_is_what_happens_when_i_go_into_fullscreen/ There's video describing his problem on that link Apparently lots of frame drops on h264 & vp9 User recently brought it up again. Posted here because it looks similar. https://www.reddit.com/r/firefox/comments/4hqle2/firefox_4601_release_notes/d2tphh9

I have bad periodic freezes during YouTube playback that only happen with 60 FPS video streams. Not only VP9, but regular mp4/h264/avc1.64 codec, too. The issue is that this phenomenon is not consistently reproducible, so my guess it is not directly issue with video playback libraries in my case, but rather with malicious synchronous-type of background GC or something like that (it kills the GUI even with e10s mode on, and even in Safe mode, so it certainly can affect video playback, too). And I know that in my case it is not related to video drivers or hardware video acceleration as I ran the same video in Chrome browser right at the time when I had issues with FireFox playing it, and it was smooth (I paused playback in FireFox, clicked play in Chrome, and then vice versa).

This is driving me nuts. MacBook Pro (Retina, 15-inch, Mid 2015) Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:46.0) Gecko/20100101 Firefox/46.0 For me the freezes generally start after scrubbing in the video a few times while in fullscreen. Reloading the page fixes the playback and I can jump to the previous time without problems. If I then scrub around the freezes start again. (Codec: avc1.64 when I bothered checking) MacBook Pro (Retina, 13-inch, Mid 2013) Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:46.0) Gecko/20100101 Firefox/46.0 When the video freezes the whole browser often becomes really unresponsive for about 20s to 2 minutes (estimate). (Old profile with tons of tabs) --- Been using Mozilla browsers since M13. For me Fx is THE browser. However this issue regularly forces me to watch videos in Chrome. If I were a more casual user of Firefox I'd probably switch away from Fx in a jiffy because of this. I think the priority of this bug needs to be upped.

(In reply to User Dderss from comment #34) > I have bad periodic freezes during YouTube playback that only happen with 60 > FPS video streams. Not only VP9, but regular mp4/h264/avc1.64 codec, too. I think you're describing an issue on Windows that is fixed in 47. (In reply to Marc Diethelm from comment #35) > This is driving me nuts. Either it is bug 1230641 or you should file a separate bug.

Richard - is the original issue still reproducible?

(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #36) > (In reply to User Dderss from comment #34) > > I have bad periodic freezes during YouTube playback that only happen with 60 > > FPS video streams. Not only VP9, but regular mp4/h264/avc1.64 codec, too. > > I think you're describing an issue on Windows that is fixed in 47. > Just updated to the latest FireFox 47 build, it is a fresh start, "about:performance" does not show any reds in either add-ons or extensions, and video stream buffer is full, yet it got the same periodical freezes (every 30 seconds or so the image in video freezes for several seconds, and then continues playing) -- as seen on this screenshot. There is simply no reason why would this happen, but it happens (not in Google Chrome, though, so it is not video card or video driver issue either).

It looks that the culprit could be the way video playback in the browser coordinates with video driver to control GPU frequency. By monitoring GPU frequency via driver's control panel, I noticed that the pattern on how the GPU frequency changes during 60 FPS video playback might be different at times when I see those massive periodical frame drops, and when the same 60 FPS video stream plays back smoothly. The GPU frequencies are not forced to higher values often enough or soon enough so it would move/process the next chunk of video frames for the hardware accelerated playback. When I click to play a 60 FPS video, my GPU frequencies spike to 880 MHz (default max for my venerable Radeon HD 6970, a top card from half a decade ago), then go down to 500 MHz, and then to 250 MHz. After some time there is short spike to 500 MHz, when, presumably, the GPU tries to catch up with the FPS that is higher than usual 30 FPS that the chip was created to accelerate video for. In normal circumstances those frequency spikes are paced properly, so 60 FPS videos play smoothly. However, in some weird circumstances -- mostly after FF freshly started, and the 60 FPS video I play is the first one -- this frequency chance pace regulation somehow does not work properly, so the GPU ends up working at lower frequencies more time than it should, so it periodically can not render chunks of frames and has to skip it. Of course, this is not something that I can be absolutely sure about, it is just my guess on how those 60 FPS freezes occur. In this case my earlier mention that the freezes are not driver issue can be wrong. (I did try to install different versions of AMD drivers, though, and it did not affect the issue.)

Anthony - per the question on Comment 37 - 1080p 60 fps VP9 plays smoothly for me on Firefox Nightly.

Mass change P2 -> P3

I'm going to close this bug on the basis that the reporter is happy that we've fixed the original issue. If there are remaining issues then please file separate bugs linking them to this bug.