Open Bug 1187118 Opened 9 years ago Updated 2 years ago

Add a GIF MediaDecoder to allow playback of animated GIFs in the <video> element

Categories

(Core :: Audio/Video: Playback, defect, P5)

defect

Tracking

()

People

(Reporter: seth, Unassigned, Mentored)

References

(Blocks 1 open bug)

Details

Attachments

(14 files, 12 obsolete files)

(deleted), patch
Details | Diff | Splinter Review
(deleted), patch
Details | Diff | Splinter Review
(deleted), patch
Details | Diff | Splinter Review
(deleted), patch
Details | Diff | Splinter Review
(deleted), patch
Details | Diff | Splinter Review
(deleted), patch
Details | Diff | Splinter Review
(deleted), patch
Details | Diff | Splinter Review
(deleted), text/x-review-board-request
Details
(deleted), text/x-review-board-request
Details
(deleted), text/x-review-board-request
Details
(deleted), text/x-review-board-request
Details
(deleted), text/x-review-board-request
Details
(deleted), text/x-review-board-request
Details
(deleted), text/x-review-board-request
Details
As part of the long term goal of playing animated GIFs through the media framework, we'd like to start by supporting them in the video element. This work should proceed behind a pref.
Mentor: seth
Depends on: 1181863
An introduction note: we are trying to move aways from the MediaDecoderReader architecture where we have one blob doing the parsing of the raw data and decoding. Instead you only need to implement a MediaDataDemuxer (bug 1154164) that will extract raw compressed samples (a MediaRawData object) and then implement a MediaDataDecoder which uncompress a single MediaRawData into a VideoData object (a picture really) The overall architecture is described in bug 1148292. The aim is to be later able to reuse the codec with different containers if need be. Probably not a worry with gif. But mostly then you benefit from all the goodness of the MediaFormatReader which provides async / highly multi-threaded decoding. So in the end, we only have a single MediaDecoderReader and a pool of demuxer and codecs.
Depends on: 1193119
(In reply to Jean-Yves Avenard [:jya] from comment #1) > We are trying to move aways from the MediaDecoderReader architecture where > we have one blob doing the parsing of the raw data and decoding. In an effort to not understate our intent here, we won't be accepting any new subclasses of MediaDecoderReader.
Priority: -- → P5
Attachment #8663972 - Flags: review?(seth)
Attachment #8663970 - Attachment is obsolete: true
Attachment #8663970 - Flags: review?(seth)
Attachment #8663972 - Attachment is obsolete: true
Attachment #8663972 - Flags: review?(seth)
Attachment #8663973 - Attachment is obsolete: true
Attachment #8663973 - Flags: review?(seth)
Attachment #8663983 - Attachment is obsolete: true
Attachment #8663983 - Flags: review?(seth)
Attachment #8663974 - Attachment is obsolete: true
Attachment #8663974 - Flags: review?(seth)
Attachment #8663977 - Attachment is obsolete: true
Attachment #8663977 - Flags: review?(seth)
Rebased version of part 1.
Assignee: mvolmer → seth
Status: NEW → ASSIGNED
Rebased part 2.
Rebased part 3.
Rebased part 4.
Assignee: seth → mvolmer
Attachment #8663995 - Attachment is obsolete: true
Attachment #8663996 - Attachment is obsolete: true
Attachment #8663999 - Attachment is obsolete: true
Attachment #8664002 - Attachment is obsolete: true
Attachment #8664004 - Attachment is obsolete: true
There are printfs in both the constructor and the destructor of the VideoData class. This way we can keep track of how many VideoData objects we have in memory at any given moment.
Attachment #8664543 - Flags: feedback?(seth)
Attachment #8664005 - Attachment is obsolete: true
Attachment #8664541 - Flags: review?(seth) → feedback?(seth)
Attachment #8664539 - Flags: review?(seth) → feedback?(seth)
Attachment #8664538 - Flags: review?(seth) → feedback?(seth)
Attachment #8664537 - Flags: review?(seth) → feedback?(seth)
Attachment #8664536 - Flags: review?(seth) → feedback?(seth)
Comment on attachment 8664543 [details] [diff] [review] (Part 6) - Keep track of how many VideoData objects are created and how many are destroyed at any given moment during playback Review of attachment 8664543 [details] [diff] [review]: ----------------------------------------------------------------- ::: dom/media/MediaData.cpp @@ +122,5 @@ > { > NS_ASSERTION(mDuration >= 0, "Frame must have non-negative duration."); > mKeyframe = aKeyframe; > mTimecode = aTimecode; > + printf("gifmediamemory %p created\n", this); We don't want to include printfs in release builds. You might want to use MOZ_LOG.
quick fly over. The image displays quickly because you set the duration of each frames that gives a refresh rate of 100Hz, causing the 77 images to be displayed in .7s only. this didn't compile for me on mac with clang (error with the gotos and variables not being initialised / defined) i was playing that GIF: http://netdna.webdesignerdepot.com/uploads7/50-inspiring-animated-gifs/006.gif it shows a black frame after a few images and the background becomes black. not sure why
(In reply to Chris Peterson [:cpeterson] from comment #21) > We don't want to include printfs in release builds. You might want to use > MOZ_LOG. This code is not ready for review.
Thank you for your feedback! (In reply to Jean-Yves Avenard [:jya] from comment #22) > The image displays quickly because you set the duration of each frames that > gives a refresh rate of 100Hz, causing the 77 images to be displayed in .7s > only. I have tried with 0.039 and it still doesn't seem to affect the frame duration. Did this value work for you? > this didn't compile for me on mac with clang (error with the gotos and > variables not being initialised / defined) This is strange because the "done" label was used before I even touched the code. I will redesign the "frame by frame" decoding code. However, I don't know what to say about the "done" label. > i was playing that GIF: > http://netdna.webdesignerdepot.com/uploads7/50-inspiring-animated-gifs/006. > gif > > it shows a black frame after a few images and the background becomes black. > > not sure why There are still some GiFs that crash. Also, loading the GIF directly seems to crash. Including them in <video> elements is working. I used this for tests: http://people.mozilla.org/~mvolmer/testpage2.html
Flags: needinfo?(jyavenard)
(In reply to Mihai Volmer [:mihaivolmer] from comment #24) > Thank you for your feedback! > > (In reply to Jean-Yves Avenard [:jya] from comment #22) > > The image displays quickly because you set the duration of each frames that > > gives a refresh rate of 100Hz, causing the 77 images to be displayed in .7s > > only. > > I have tried with 0.039 and it still doesn't seem to affect the frame > duration. Did this value work for you? it did yes (for the gif I was using) > > > this didn't compile for me on mac with clang (error with the gotos and > > variables not being initialised / defined) > > This is strange because the "done" label was used before I even touched the > code. I will redesign the "frame by frame" decoding code. However, I don't > know what to say about the "done" label. it's the goto finished_frame; that gave error likely because bool hitFinishedFrame = false; is defined after the goto, but before the label. moving + bool hitFinishedFrame = false; to be before the loop will likely do the trick too (still ugly imho)
Flags: needinfo?(jyavenard)
(In reply to Jean-Yves Avenard [:jya] from comment #25) > moving + bool hitFinishedFrame = false; to be before the loop will likely > do the trick too > > (still ugly imho) That code was just a hack to get things to work for the demo. I wouldn't r+ it. In fact, we're probably going to need to semi-rewrite the GIF decoder before we actually land anything in this bug. (See bug 1204392.)
Attachment #8664536 - Flags: feedback?(seth)
Attachment #8664537 - Flags: feedback?(seth)
Attachment #8664538 - Flags: feedback?(seth)
Attachment #8664539 - Flags: feedback?(seth)
Attachment #8664541 - Flags: feedback?(seth)
Attachment #8664543 - Flags: feedback?(seth)
Mihai, I'm clearing these feedback flags because I think we need to make progress on getting basic functionality working before we're ready for detailed feedback. At this time I think the biggest concern is fixing the frame duration problem.
Blocks: 1247520
Blocks: 1257388
No longer blocks: 1247520
Depends on: 1204392
No longer depends on: 1193119
Depends on: 1337111

Mihai, are you still working on this or should we unassign it?
Looks like the dependent bugs are now fixed.

Flags: needinfo?(mihaivolmer)

It's been 2 months without response, perhaps Mihai is MIA? (or needs another ping?)

Assignee: mihaivolmer → nobody
Status: ASSIGNED → NEW
Flags: needinfo?(mihaivolmer)
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: