Found while fuzzing m-c 20221101-f8dff2edfe1b (--enable-address-sanitizer --enable-fuzzing)

To reproduce via Grizzly Replay:

$ pip install fuzzfetch grizzly-framework
$ python -m fuzzfetch -a --fuzzing -n firefox
$ python -m grizzly.replay ./firefox/firefox testcase.webm

UndefinedBehaviorSanitizer: gecko/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp:995:19: runtime error: load of value 191, which is not a valid value for type 'enum AVColorSpace'

/builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp:995:19: runtime error: load of value 191, which is not a valid value for type 'enum AVColorSpace'
    #0 0x7f96e9437852 in mozilla::FFmpegVideoDecoder<46465650>::GetFrameColorSpace() const /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp:995:19
    #1 0x7f96e9436609 in mozilla::FFmpegVideoDecoder<46465650>::CreateImage(long, long, long, nsTArray<RefPtr<mozilla::MediaData> >&) const /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp:1119:22
    #2 0x7f96e943371a in mozilla::FFmpegVideoDecoder<46465650>::DoDecode(mozilla::MediaRawData*, unsigned char*, int, bool*, nsTArray<RefPtr<mozilla::MediaData> >&) /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp:915:12
    #3 0x7f96e94275fd in mozilla::FFmpegDataDecoder<46465650>::DoDecode(mozilla::MediaRawData*, bool*, nsTArray<RefPtr<mozilla::MediaData> >&) /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataDecoder.cpp:193:10
    #4 0x7f96e9426da1 in mozilla::FFmpegDataDecoder<46465650>::ProcessDecode(mozilla::MediaRawData*) /builds/worker/checkouts/gecko/dom/media/platforms/ffmpeg/FFmpegDataDecoder.cpp:147:20
    #5 0x7f96e943d68b in applyImpl<mozilla::FFmpegDataDecoder<46465650>, RefPtr<mozilla::MozPromise<nsTArray<RefPtr<mozilla::MediaData> >, mozilla::MediaResult, true> > (mozilla::FFmpegDataDecoder<46465650>::*)(mozilla::MediaRawData *), StoreRefPtrPassByPtr<mozilla::MediaRawData>, 0UL> /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1147:12
    #6 0x7f96e943d68b in apply<mozilla::FFmpegDataDecoder<46465650>, RefPtr<mozilla::MozPromise<nsTArray<RefPtr<mozilla::MediaData> >, mozilla::MediaResult, true> > (mozilla::FFmpegDataDecoder<46465650>::*)(mozilla::MediaRawData *)> /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1153:12
    #7 0x7f96e943d68b in mozilla::detail::MethodCall<mozilla::MozPromise<nsTArray<RefPtr<mozilla::MediaData> >, mozilla::MediaResult, true>, RefPtr<mozilla::MozPromise<nsTArray<RefPtr<mozilla::MediaData> >, mozilla::MediaResult, true> > (mozilla::FFmpegDataDecoder<46465650>::*)(mozilla::MediaRawData*), mozilla::FFmpegDataDecoder<46465650>, mozilla::MediaRawData*>::Invoke() /builds/worker/workspace/obj-build/dist/include/mozilla/MozPromise.h:1518:47
    #8 0x7f96e943d19d in mozilla::detail::ProxyRunnable<mozilla::MozPromise<nsTArray<RefPtr<mozilla::MediaData> >, mozilla::MediaResult, true>, RefPtr<mozilla::MozPromise<nsTArray<RefPtr<mozilla::MediaData> >, mozilla::MediaResult, true> > (mozilla::FFmpegDataDecoder<46465650>::*)(mozilla::MediaRawData*), mozilla::FFmpegDataDecoder<46465650>, mozilla::MediaRawData*>::Run() /builds/worker/workspace/obj-build/dist/include/mozilla/MozPromise.h:1538:42
    #9 0x7f96e274deb6 in mozilla::TaskQueue::Runner::Run() /builds/worker/checkouts/gecko/xpcom/threads/TaskQueue.cpp:259:20
    #10 0x7f96e277a642 in nsThreadPool::Run() /builds/worker/checkouts/gecko/xpcom/threads/nsThreadPool.cpp:309:14
    #11 0x7f96e276c93e in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:1198:16
    #12 0x7f96e2776bc4 in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/checkouts/gecko/xpcom/threads/nsThreadUtils.cpp:465:10
    #13 0x7f96e3f08195 in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/checkouts/gecko/ipc/glue/MessagePump.cpp:300:20
    #14 0x7f96e3d862f1 in RunInternal /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:381:10
    #15 0x7f96e3d862f1 in RunHandler /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:374:3
    #16 0x7f96e3d862f1 in MessageLoop::Run() /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:356:3
    #17 0x7f96e2763a98 in nsThread::ThreadFunc(void*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:383:10
    #18 0x7f97051e93ee in _pt_root /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:201:5
    #19 0x7f9705ea4608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8608) (BuildId: 7b4536f41cdaa5888408e82d0836e33dcf436466)
    #20 0x7f9705a6b132 in clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)

This test case also triggers the following assertion on a debug build:

Assertion failure: wroteSequenceHeader, at /builds/worker/checkouts/gecko/dom/media/platforms/wrappers/MediaChangeMonitor.cpp:384

#0 0x7f7fb322d40d in mozilla::AV1ChangeMonitor::UpdateConfig(mozilla::AOMDecoder::AV1SequenceInfo const&) /builds/worker/checkouts/gecko/dom/media/platforms/wrappers/MediaChangeMonitor.cpp:384:5
#1 0x7f7fb322d7fb in mozilla::AV1ChangeMonitor::CheckForChange(mozilla::MediaRawData*) /builds/worker/checkouts/gecko/dom/media/platforms/wrappers/MediaChangeMonitor.cpp:425:5
#2 0x7f7fb3222c1d in mozilla::MediaChangeMonitor::CreateDecoderAndInit(mozilla::MediaRawData*) /builds/worker/checkouts/gecko/dom/media/platforms/wrappers/MediaChangeMonitor.cpp:723:36
#3 0x7f7fb3222002 in mozilla::MediaChangeMonitor::CheckForChange(mozilla::MediaRawData*) /builds/worker/checkouts/gecko/dom/media/platforms/wrappers/MediaChangeMonitor.cpp:835:12
#4 0x7f7fb3221994 in mozilla::MediaChangeMonitor::Decode(mozilla::MediaRawData*) /builds/worker/checkouts/gecko/dom/media/platforms/wrappers/MediaChangeMonitor.cpp:546:20
#5 0x7f7fb3237f7a in operator() /builds/worker/checkouts/gecko/dom/media/platforms/wrappers/MediaDataDecoderProxy.cpp:31:33
#6 0x7f7fb3237f7a in mozilla::detail::ProxyFunctionRunnable<mozilla::MediaDataDecoderProxy::Decode(mozilla::MediaRawData*)::$_18, mozilla::MozPromise<nsTArray<RefPtr<mozilla::MediaData> >, mozilla::MediaResult, true> >::Run() /builds/worker/workspace/obj-build/dist/include/mozilla/MozPromise.h:1645:29
#7 0x7f7faf1a3245 in mozilla::TaskQueue::Runner::Run() /builds/worker/checkouts/gecko/xpcom/threads/TaskQueue.cpp:259:20
#8 0x7f7faf1bf3af in nsThreadPool::Run() /builds/worker/checkouts/gecko/xpcom/threads/nsThreadPool.cpp:309:14
#9 0x7f7faf1b6314 in nsThread::ProcessNextEvent(bool, bool*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:1198:16
#10 0x7f7faf1bca5d in NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/checkouts/gecko/xpcom/threads/nsThreadUtils.cpp:465:10
#11 0x7f7fafdb9adb in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/checkouts/gecko/ipc/glue/MessagePump.cpp:300:20
#12 0x7f7fafcdbf77 in MessageLoop::RunInternal() /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:381:10
#13 0x7f7fafcdbe82 in RunHandler /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:374:3
#14 0x7f7fafcdbe82 in MessageLoop::Run() /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:356:3
#15 0x7f7faf1b16c6 in nsThread::ThreadFunc(void*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:383:10
#16 0x7f7fc3c20e27 in _pt_root /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:201:5
#17 0x7f7fc44cbb42 in start_thread nptl/./nptl/pthread_create.c:442:8
#18 0x7f7fc455d9ff  misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Verified bug as reproducible on mozilla-central 20221102174350-6d65bca9434c.
The bug appears to have been introduced in the following build range:

Start: 7d3600925e24a1c8cf634968d0afa43e41e00d1d (20220329114347)
End: ac056c06d8cac6a625c33f5d3e003548ccd2ec57 (20220329130731)
Pushlog: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=7d3600925e24a1c8cf634968d0afa43e41e00d1d&tochange=ac056c06d8cac6a625c33f5d3e003548ccd2ec57

:stransky could this be related to bug 1761471?

Looks like the AV1 bitstream itself carries invalid CICP color space values, this is what I see after patching AOMDecoder::ReadSequenceHeaderInfo() to fix the previously mentioned assertion failure (Bug 1799787):
{mPrimaries=179 '³' mTransfer=248 'ø' mMatrix=191 '¿' ...}

This means that after the patch in that bug lands, this bug's error will likely move to that function instead, as the enum values will be found to be invalid there before any decoding takes place. That patch also causes the full sequence header to be read instead, and that allows it to detect the fuzzing testcase as a corrupted header since the trailing zero bits are non-zero.

I'm unsure if the invalid enum values are actually an issue, though, since decoding/rendering specifically only handles the enum values that are implemented, and any others are either defaulted when they're used or cause a decoder error. If we do need to validate the color space values, it would be good for it to be done in centralized functions, so that if more color space values are added to the standard later, they can be easily allowed through the rendering pipeline. Not sure how those functions could be used in cases like this where it needs to validate an enum defined by a library, though.

Any chance we could rerun this to see what impact bug 1799787 has had?

The attached test case no longer reproduces the issue. Tested with m-c 20221115-f130aa968d7e.

Testcase crashes using the initial build (mozilla-central 20221101213659-f8dff2edfe1b) but not with tip (mozilla-central 20221118154632-3b5a8f67189b.)

The bug appears to have been fixed in the following build range:

Start: 706788071c3635ff199f4a2505c0308f478ad0cf (20221110200126)
End: 55fc19ba0ec4aa750e72c4b9cb3f9bbc55e6f6b8 (20221110213727)
Pushlog: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=706788071c3635ff199f4a2505c0308f478ad0cf&tochange=55fc19ba0ec4aa750e72c4b9cb3f9bbc55e6f6b8

tsmith, can you confirm that the above bisection range is responsible for fixing this issue?
Removing bugmon keyword as no further action possible. Please review the bug and re-add the keyword for further analysis.