Firefox Nightly 96.0a1 20211204215420, 32-bit version

https://council.gwangju.go.kr/index.do crashes trying to load a web font. Deian, could you take a look?

Crash report: https://crash-stats.mozilla.org/report/index/ce9bd866-e349-4b3c-a7d6-320730211205

Reason: EXCEPTION_ACCESS_VIOLATION_WRITE

Top 10 frames of crashing thread:

0 xul.dll w2c_memset security/rlbox/rlbox.wasm.c:661399
1 xul.dll w2c_RLBoxConvertWOFF2ToTTF security/rlbox/rlbox.wasm.c:229877
2 xul.dll rlbox::rlbox_sandbox<rlbox::rlbox_wasm2c_sandbox>::INTERNAL_invoke_with_func_ptr<bool  third_party/rlbox/include/rlbox_sandbox.hpp:790
3 xul.dll RLBoxProcessWOFF2 gfx/ots/RLBoxWOFF2Host.cpp:168
4 xul.dll ots::OTSContext::Process gfx/ots/src/ots.cc:1084
5 xul.dll gfxOTSMessageContext::Process gfx/thebes/gfxUserFontSet.cpp:168
6 xul.dll gfxUserFontEntry::SanitizeOpenTypeData gfx/thebes/gfxUserFontSet.cpp:199
7 xul.dll gfxUserFontEntry::StartPlatformFontLoadOnBackgroundThread gfx/thebes/gfxUserFontSet.cpp:637
8 xul.dll static mozilla::detail::RunnableMethodArguments<unsigned int, const unsigned char*, unsigned int, nsMainThreadPtrHandle<nsIFontLoadCompleteCallback> >::applyImpl<gfxUserFontEntry, void  xpcom/threads/nsThreadUtils.h:1147
9 xul.dll mozilla::detail::RunnableMethodImpl<gfxUserFontEntry*, void  xpcom/threads/nsThreadUtils.h:1200

Upon investigation, it looks like this is an oom ... this happened when decompressing a font whose decompressed size is 17mb (which OOMs the 16MB sandbox default on 32-bit platforms)

1. this call here https://searchfox.org/mozilla-central/source/gfx/ots/RLBoxWOFF2Host.cpp#170 passes in a decompressed size of 17mb
2. this causes this malloc in the sandbox to fail https://searchfox.org/mozilla-central/source/modules/woff2/RLBoxWOFF2Sandbox.cpp#15

We should try to estimate the size of the sandbox needed and create a sandbox of that size. The way to estimate the decompressed size (which is actually done in the same file https://searchfox.org/mozilla-central/source/gfx/ots/RLBoxWOFF2Host.cpp#152) is shown by another woff2 function. We need to do this estimation outside the sandbox and compute the estimate before sandbox creation

Working on a fix

Going to do this in parts

Part 1 - Move the mem estimation code outside the sandbox

• From here modules/woff2/src/woff2_dec.cc:1300

Part 2 - Augment the RLBoxSandboxPool

• RLBoxSandboxPool::CreateSandboxData function accept params that it can pass to the RLBoxWoff2SandboxPool::CreateSandboxData
• RLBoxSandboxPool::PopOrCreate needs to accept a function pointer that accepts a "sandbox filter" that would apply a filter condition (Filter's type would be bool (*)(RLBoxSandboxPoolData&))

Part 3 - Write an algorithm to estimate size of sandbox based on size of decompressed font. Need to do some testing to figure out what this estimate should be. But maybe something like at 50% more that the expected decompress size to account for input size and misc allocs

• I need to add a small feature to RLBox to expose the maximum memory of a sandbox (it has only the "current memory" right now)

Simplifying this, we can combine parts 2 and 3 above and skip the rlbox update

Set release status flags based on info from the regressing bug 1743324

Backed out for causing build bustages at RLBoxSandboxPool.cpp

Backout link: https://hg.mozilla.org/integration/autoland/rev/935f867d7bdaeeed38e25ad501479710cf4c759e

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&selectedTaskRun=f8t83fF-R7u1eaCZE4f3bg.0&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=582101d582a02adfbaf0fda4d5c15d38f26fab65

Failure log: https://treeherder.mozilla.org/logviewer?job_id=360550960&repo=autoland&lineNumber=59230

Have updated the patch to fix this. Will be landed shortly.

Should the dropped nsExpatDriver crash signatures get a new bug?

https://hg.mozilla.org/mozilla-central/rev/693727e2df8b
https://hg.mozilla.org/mozilla-central/rev/f21aa87c2301

Sorry, could you clarify your question? Is there something wrong with crash signatures?

(In reply to Shravan Narayan from comment #13)

Sorry, could you clarify your question? Is there something wrong with crash signatures?

Sebastian added some new crash signatures right before comment 6, and you removed them (probably unintentionally) in comment 9.

Sebastian, please do file a separate bug for those.

The crash reason for those is "RLBox crash: Called memcpy for memory larger than the sandbox", which suggests we may need to apply a similar trick for expat.

The patch landed in nightly and beta is affected.
:shravanrn, is this bug important enough to require an uplift?
If not please set status_beta to wontfix.

For more information, please visit auto_nag documentation.

Comment on attachment 9254226 [details]
Bug 1744460 part 2 - Update woff2 RLBoxSandboxPool to track minimum sandbox size r=bholley

Beta/Release Uplift Approval Request

• User impact if declined: Content Process crashes on 32-bit builds loading certain rare large fonts.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Crashes fixed on Nightly. Patches have some light refactoring, but mostly just ensure that the sandbox is big enough.
• String changes made/needed:

Comment on attachment 9254226 [details]
Bug 1744460 part 2 - Update woff2 RLBoxSandboxPool to track minimum sandbox size r=bholley

Approved for 96.0b5

https://hg.mozilla.org/releases/mozilla-beta/rev/488aad6dab04
https://hg.mozilla.org/releases/mozilla-beta/rev/c5eccd70bae9

I reproduced the crash with the signature reported here [@ w2c_memset] using old Nightly build https://crash-stats.mozilla.org/report/index/6f90c406-ce87-4bc5-94f8-99e9a0211221.
Using latest Nightly (97.0a1) and Beta (96.0b7) builds I still get a crash using steps from comment 0 but with a different crash signature [@ moz_wasm2c_trap_handler], not sure if they are related or not. Shravanrn could you please take a look at these crashes?

https://crash-stats.mozilla.org/report/index/1ac73c92-cb61-4c29-b9a3-cd0070211221 - Nightly
https://crash-stats.mozilla.org/report/index/ee216d5b-9c20-4455-8227-bcb8f0211221 - Beta

I am able to repro the crash on the latest nightly. Investigating now

Closing this bug, moving the new report to bug 1747586.

Closing based on the last comments. The remaining issue is addressed in bug 1747586.
Furthermore, the original crash signature is no longer reproducing in fixed builds (Nightly builds after v97.0a1 from 2021-12-21 or Beta 96.0b7).