In bug 1750858, we worked around the fact that mPicSize in PlanarYCbCrData was never properly supported since the mYSize/mCbCrSize ratios were required to specify the amount of chroma subsampling in the YCbCr data. In the event of any alignment of these sizes, it was no longer possible to determine that at all, so that while mPicSize can determine the display rectangle for the Y data, it couldn't be relied upon for the CbCr data. While looking at the sizes can be flexible, it is finicky and error prone downstream for things that need to determine formats for manipulating the data.

After some initial discussion, I thought it might be beneficial to explicitly specify the chroma subsampling scheme in the data, that way regardless of the size/alignment/padding/etc. of the data, we can always determine how to map the data to an appropriate YCbCr processing routine.

Putting up a patch here as a proof of concept to engender discussion about how we might want to proceed, if at all. Nomenclature, mechanisms, etc. are all subject to debate, but I just wanted a proof of concept to see the scope of the changes necessary to get this working.

Backed out changeset 36c6350bc7d1 (bug 1757067) for causing build bustages in gfx/gl/GLBlitHelperD3D.cpp

Backout link: https://hg.mozilla.org/integration/autoland/rev/ad5e9f350f67805fbd3e31643f1988255d4e8286

Push with failures

Failure log

Failure log #2

INFO -  /builds/worker/fetches/sccache/sccache /builds/worker/fetches/clang/bin/clang-cl -Xclang -std=c++17 -FoUnified_cpp_gfx_gl0.obj -c  -I/builds/worker/workspace/obj-build/dist/stl_wrappers -guard:cf -U_FORTIFY_SOURCE -Xclang -fno-common -DNDEBUG=1 -DTRIMMED=1 -DUNICODE -D_UNICODE -D_CRT_RAND_S -DCERT_CHAIN_PARA_HAS_EXTRA_FIELDS -D_SECURE_ATL -DCHROMIUM_BUILD -DU_STATIC_IMPLEMENTATION -DOS_WIN=1 -DWIN32 -D_WIN32 -D_WINDOWS -DWIN32_LEAN_AND_MEAN -DCOMPILER_MSVC -DMOZ_D3DCOMPILER_VISTA_DLL=d3dcompiler_47.dll -DWINAPI_NO_BUNDLED_LIBRARIES -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -DSTATIC_EXPORTABLE_JS_API -I/builds/worker/checkouts/gecko/gfx/gl -I/builds/worker/workspace/obj-build/gfx/gl -I/builds/worker/workspace/obj-build/ipc/ipdl/_ipdlheaders -I/builds/worker/checkouts/gecko/ipc/chromium/src -I/builds/worker/checkouts/gecko/gfx/skia -I/builds/worker/checkouts/gecko/gfx/skia/skia -I/builds/worker/checkouts/gecko/gfx/cairo/cairo/src -I/builds/worker/workspace/obj-build/dist/include -I/builds/worker/workspace/obj-build/dist/include/nspr -I/builds/worker/workspace/obj-build/dist/include/nss -MD -FI /builds/worker/workspace/obj-build/mozilla-config.h -DMOZILLA_CLIENT -Qunused-arguments -Qunused-arguments -fsanitize=address -fsanitize-blacklist=/builds/worker/checkouts/gecko/build/sanitizers/asan_blacklist_win.txt -fcrash-diagnostics-dir=/builds/worker/artifacts -fcrash-diagnostics-dir=/builds/worker/artifacts -fcrash-diagnostics-dir=/builds/worker/artifacts -TP -Zc:sizedDealloc- -D_HAS_EXCEPTIONS=0 -W3 -Gy -Zc:inline -Wno-inline-new-delete -Wno-invalid-offsetof -Wno-microsoft-enum-value -Wno-microsoft-include -Wno-unknown-pragmas -Wno-ignored-pragmas -Wno-deprecated-declarations -Wno-invalid-noreturn -Wno-inconsistent-missing-override -Wno-implicit-exception-spec-mismatch -Wno-microsoft-exception-spec -Wno-unused-local-typedef -Wno-ignored-attributes -Wno-used-but-marked-unused -D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING -GR- -Z7 -Xclang -load -Xclang /builds/worker/workspace/obj-build/build/clang-plugin/libclang-plugin.so -Xclang -add-plugin -Xclang moz-check -O2 -gline-tables-only -Oy- -Werror -Werror=switch -fno-strict-aliasing  -Xclang -MP -Xclang -dependency-file -Xclang .deps/Unified_cpp_gfx_gl0.obj.pp -Xclang -MT -Xclang Unified_cpp_gfx_gl0.obj   Unified_cpp_gfx_gl0.cpp
[task 2022-03-10T08:04:46.427Z] 08:04:46     INFO -  In file included from Unified_cpp_gfx_gl0.cpp:29:
[task 2022-03-10T08:04:46.427Z] 08:04:46     INFO -  /builds/worker/checkouts/gecko/gfx/gl/GLBlitHelperD3D.cpp(213,68): error: no member named 'YDataSize' in 'mozilla::layers::D3D11YCbCrImage'
[task 2022-03-10T08:04:46.428Z] 08:04:46     INFO -    return BlitAngleYCbCr(handles, srcImage->mPictureRect, srcImage->YDataSize(),
[task 2022-03-10T08:04:46.428Z] 08:04:46     INFO -                                                           ~~~~~~~~  ^
[task 2022-03-10T08:04:46.428Z] 08:04:46     INFO -  /builds/worker/checkouts/gecko/gfx/gl/GLBlitHelperD3D.cpp(214,35): error: no member named 'CbCrDataSize' in 'mozilla::layers::D3D11YCbCrImage'
[task 2022-03-10T08:04:46.428Z] 08:04:46     INFO -                          srcImage->CbCrDataSize(), srcImage->mColorSpace,
[task 2022-03-10T08:04:46.428Z] 08:04:46     INFO -                          ~~~~~~~~  ^
[task 2022-03-10T08:04:46.429Z] 08:04:46     INFO -  2 errors generated.

https://hg.mozilla.org/mozilla-central/rev/6b2d2aa11145