https://webglsamples.org/aquarium/aquarium.html reports that "it does not appear your computer supports WebGL".

$ ./mach run
Crash Annotation GraphicsCriticalError: |[0][GFX1-]: Failed GL context creation for WebRender: 0x0 (t=0.939288) [GFX1-]: Failed GL context creation for WebRender: 0x0
Crash Annotation GraphicsCriticalError: |[0][GFX1-]: Failed GL context creation for WebRender: 0x0 (t=0.939288) |[1][GFX1-]: Failed to connect WebRenderBridgeChild. (t=0.939479) [GFX1-]: Failed to connect WebRenderBridgeChild.

This does not appear to be sandboxing related as I can reproduce the problem with the content sandbox disabled.

Reproduced on a MacBook Air (11-inch, Early 2015) and also reported on bug 1647816. On Firefox 77.0.1 and Nightly revision d25dfe67b69e.

This looks like a case of PR_LoadLibrary() failing due to the new linker cache and dynamic library behavior in Big Sur documented below. More specifically, the call to PR_LoadLibrary() in CGLLibrary::EnsureInitialized() is failing to load /System/Library/Frameworks/OpenGL.framework/OpenGL because the access(2) call in PR_Access() returns -1.

From https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11-beta-release-notes/

New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker
cache of all system-provided libraries. As part of this change, copies of dynamic
libraries are no longer present on the filesystem. Code that attempts to check
for dynamic library presence by looking for a file at a path or enumerating a
directory will fail. Instead, check for library presence by attempting to dlopen()
the path, which will correctly check for the library in the cache. (62986286)

I'll try a patch that skips the access() check and report back the results.

With a patch to pr_LoadLibraryByPathname() to skip checking for access to the library on the filesystem, WebGL appears to work on Big Sur. Taking the bug.

The severity field is not set for this bug.
:jgilbert, could you have a look please?

For more information, please visit auto_nag documentation.

This PR_Access() call was added for bug 480730 back in 2009.

@Kai, I'd like to get this fix landed to address the problem described on comment 2 for macOS Big Sur, but don't have any experience with nspr. Is this something we could land in nspr and then pull into mozilla-central for an uplift to Release and ESR? The earlier we can get this fixed, the more time we have to qualify Firefox on macOS Big Sur Betas before it ships.

For mozilla-central 80, we're already intending to release a new NSPR 4.27 (bug 1652330).
As soon as the patch is reviewed, I will check it into the main NSPR repository, and then I will uplift a new NSPR nightly snapshot into mozilla-central for FF 80.

ESR 78 uses NSPR 4.25

Once you confirm the new NSPR snapshot works with FF 80, I can create a NSPR 4.25.1 patch release that picks up the patch.

FF 79 uses NSPR 4.26, so we'd require a 4.26.1 patch release, or we wontfix FF 79.

Let's use bug 1652956 to track the code change to NSPR.

Let's use this bug to track uplifting of NSPR snapshots into firefox branches.

(In reply to Kai Engert (:KaiE:) from comment #9)

Let's use bug 1648836 to track the code change to NSPR.

Sorry, wrong number, should have been bug 1652956. (fixed in previous comment)

The uplift of the NSPR fix to mozilla-central for FF 80 is tracked in bug 1652330.

Would it be possible to create a list of all potential system PATH prefixes?

I'm just guessing, but could we assume that the path to all system libraries will be prefixed by either /usr/lib/ or /System/ ?

The change to NSPR has uncovered a regression in NSS unfortunately. While it seems easy to fix the regression in NSS, it's not ideal to uplift the change of behavior to old stable NSPR branches. It would require all users of NSPR to also pick up a corresponding NSS fix release.

If we can identify a common path prefix for all system libraries on macOS (that are affected by the change of behavior on macOS 11), then we could change NSPR to skip the check on input with that prefix, only. By keeping the file check for other locations, and continuing to fail it the file isn't found for other locations, we could avoid the change of behavior.

Haik, could you please test if this patch applied on top of mozilla-central still fixes this bug?

fixed a typo

(In reply to Kathleen :Brade from comment #17)

Comment on attachment 9164858 [details] [diff] [review]
1648836-v4.patch

I believe that this line:
+ const char *systemPrefixLen = strlen(systemPrefix);
was intended to be something like:
+ const size_t systemPrefixLen = strlen(systemPrefix);

Correct, thanks a lot for catching this bug!

To be extra careful (in case name is shorter than systemPrefix), I would
change this line:
+ if (strncmp(name, systemPrefix, systemPrefixLen) == 0) {
to something like:
+ if ((strlen(name) > systemPrefixLen) && strncmp(name, systemPrefix, systemPrefixLen) == 0) {

Fully agreed. thanks

(In reply to Kai Engert (:KaiE:) from comment #13)

Would it be possible to create a list of all potential system PATH prefixes?

I'm just guessing, but could we assume that the path to all system libraries will be prefixed by either /usr/lib/ or /System/ ?

I think that's a good assumption for now. I've looked on 10.15 and 11.0, but will check this makes sense for some older OS releases too. More info:

I haven't found any documentation about exactly which libraries are affected by the change in macOS 11, just "system" libraries. /usr/lib and /System should only contain system libraries and that is supported by how starting with macOS 10.15, both /System and /usr are on a read-only filesystem.

Are system libraries installed outside of those directories? There are some libraries in /Library that ship with the system, but these are not on a read-only filesystem and are not broken symlinks (like is the case with the missing dylibs in /System and /usr/lib). So it appears that it is safe to do a filesystem check on /Library for now. I also used the dyld_shared_cache_util tool (from the Apple opensource site) to query the shared library cache on Big Sur and it only contains libraries from /System and /usr/lib.

And the docs on the Mac filesystem[1] support that these are the system library parent directories.

If we can identify a common path prefix for all system libraries on macOS (that are affected by the change of behavior on macOS 11), then we could change NSPR to skip the check on input with that prefix, only. By keeping the file check for other locations, and continuing to fail it the file isn't found for other locations, we could avoid the change of behavior.

This sounds like a good solution to me. We should also fix this issue in NSS anyway so we can remove the path prefix checks in a later release.

1. https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW15

Haik, thanks for the feedback.

This updated patch will treat both /System/ and /usr/lib/ prefixes as system libraries.

(In reply to Kai Engert (:KaiE:) from comment #21)

This updated patch will treat both /System/ and /usr/lib/ prefixes as system libraries.

The name_len > sytemPrefixLen[1,2] checks are not needed.

The more restrictive fix, which should avoid the regression, will land later today with bug 1652330.

After it lands, please test if the issue is still fixed.

I verified that WebGL on Big Sur works and the OpenGL library is loaded successfully with the v8 patch. Here's a clean macOS try push https://treeherder.mozilla.org/#/jobs?repo=try&selectedTaskRun=CTvMaw6lSoeWZkr_vs1PFA.0&revision=3428c9a0908497cae2b936245e58521cb2274b1b

Thanks. The js-ctypes issue from bug 1648352 has been confirmed to be fixed, too.

Let's wait for tomorrow. If we don't get any regression reports, I'll prepare uplifts for ESR 68 and ESR 78 and will request approval.

WebGL on 80.0a1 (2020-07-22) (64-bit) Nightly running on Big Sur Beta 3 is working as expected per test case in Comment 1.

NSPR 4.27 RTM released an uplift to m-c requested in bug 1652330.

NSPR 4.25.1 backport for esr78 commited, esr78 try build running here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=8063aa5048ff351bac473fa63c8536095b682c11

NSPR 4.21.1 backport for esr68 commited, esr68 try build running here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c169d64b02458fe8307e2bbb496ebcf415ecbf6c

(try builds are mac only, because all code changes are inside #ifdef DARWIN )

Once try builds are green, I'll finalize the 4.25.1 and 4.21.1 releases and request uplift to esr68 and esr78

Comment on attachment 9165735 [details]
Bug 1648836 - Uplift NSPR 4.25.1 to support macOS 11. r=jcj

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: support macOS 11
• User impact if declined: FF 78.x not working on macOS 11
• Fix Landed on Version: 80
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Limited to macOS platform. Only affects loading of library files from /System/ and /usr/lib/. Should Firefox attempt to load a library from a system location, but that library doesn't exist in that exact location, but a library with the same name exists elsewhere on the system, then loading will succeed. It's only a problem if Firefox expects that the loading of the system library in a nonexisting location returns a failure. It seems unlikely that Firefox depends on this behavior for system libraries.
• String or UUID changes made by this patch: none

Comment on attachment 9165734 [details]
Bug 1648836 - Uplift NSPR 4.21.1 to support macOS 11. r=jcj

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: support macOS 11
• User impact if declined: FF 68.x not working on macOS 11
• Fix Landed on Version: 80
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Limited to macOS platform. Only affects loading of library files from /System/ and /usr/lib/. Should Firefox attempt to load a library from a system location, but that library doesn't exist in that exact location, but a library with the same name exists elsewhere on the system, then loading will succeed. It's only a problem if Firefox expects that the loading of the system library in a nonexisting location returns a failure. It seems unlikely that Firefox depends on this behavior for system libraries.
• String or UUID changes made by this patch: none

@jcristau, let's discuss our plan for supporting Big Sur on the ESR's before landing the ESR 68 fix. The release date for ESR 68.12 is 2020-08-25 and we have a growing list of macOS Big Sur bugs (meta bug 1648487). We don't know the release date for Big Sur, but if it ends up being after the switch over to ESR 78.2, uplifts to ESR 68 would not be necessary.

Marking this bug as resolved. The problem was addressed by Kai's NSPR patch which is in Firefox 80.

Comment on attachment 9165735 [details]
Bug 1648836 - Uplift NSPR 4.25.1 to support macOS 11. r=jcj

Approved for 78.2esr.

https://hg.mozilla.org/releases/mozilla-esr78/rev/144ac8f6c028

Comment on attachment 9165734 [details]
Bug 1648836 - Uplift NSPR 4.21.1 to support macOS 11. r=jcj

approved for 68.12esr

https://hg.mozilla.org/releases/mozilla-esr68/rev/e709a355a4ff

We managed to take a look on older versions of macOS (10.9->10.15) for both the 68/78 esr builds.
The smoke session for the older builds can be found in this document.

Upgrading one test-machine to the public 11.0 macOS, revealed several issues that we're still documenting.
So in regards to this, we will wait until next week until we update the flags; so we can properly asses the status of the patch(es).

Marking as verified after confirming patch with macOS 11.0 as well.
The detailed report can be found here.
All of the encountered issues don't appear to be related to this patch and we filed separate bugs for each.