OOC, how long does that final 'xcrun' command take? Does it modify Firefox.app.zip directly?

(In reply to Chris AtLee [:catlee] from comment #6)

OOC, how long does that final 'xcrun' command take? Does it modify Firefox.app.zip directly?

It takes 2-3 minutes for me with a zip file that is 76MB. It doesn't modify the zip file.

That is not the time it takes for the approval to occur, just to get the request submitted.

I just did a test and the zip file was listed as accepted in less than two minutes after the --notarize-app command completed.

There's one more command that needs to be run after the acceptance which completes within a few seconds:

$ xcrun stapler staple -v Nightly.app

I'm not hitting the crash I mentioned in Comment 5. It was just a matter of setting the appropriate entitlements. I'm working on determining what entitlements we would want to use for Firefox builds and plan to land an entitlement file that releng would use when running the codesign and xcrun commands.

Nick and I are looking at automating this; first we'll need to successfully notarize manually.

Did you set up a developer account? We may do so for testing, to avoid using the production account.
Did you recompile, or just specify a different --primary-bundle-id to test?
Do we need new builds every time we re-submit for testing? (We're guessing re-signing may suffice, since that will change the contents, but we're not sure.)

(Just asking to save time if you already have the answers here.)

A hacky bash script for testing Notarization:

$ ./codesign-bundle-hardened.bash
Usage: ./codesign-bundle-hardened.bash
-a <PATH-TO-BROWSER.app>
-i <IDENTITY>
-e <ENTITLEMENTS-FILE>
[-o <OUTPUT-ZIP-FILE>]

$ ./codesign-bundle-hardened.bash -a ./Nightly.app
-i <YOUR_SIGNING_IDENTITY>
-e ./browser.entitlements
-o Nightly.app.zip

$ xcrun altool --notarize-app
-f Nightly.app.zip
--primary-bundle-id "org.haikaftandilian.nightly"
-u <APPLE_DEVELOPER_ID> # your apple developer ID email address
-p "@keychain:AC_PASSWORD"

Wait for email regarding notarization acceptance

$ xcrun stapler staple -v Nightly.app

(In reply to Aki Sasaki [:aki] from comment #9)

Nick and I are looking at automating this; first we'll need to successfully notarize manually.

Did you set up a developer account? We may do so for testing, to avoid using the production account.

I already had a developer account and followed some steps online to setup the cert. I know it's possible to create alternate (e.g., developer or testing) certificates for the same account so you could use a new testing cert on the production account, but I think you're saying you'd rather avoid using the production account entirely.

Did you recompile, or just specify a different --primary-bundle-id to test?

I've tested with my own build from central and also with re-signing an official release build.

Do we need new builds every time we re-submit for testing? (We're guessing re-signing may suffice, since that will change the contents, but we're not sure.)

I've been resubmitting the same build after re-running the codesign commands. My script uses the codesign --force option to overwrite existing signatures. I've found that submitting a duplicate zip file is detected and blocked, but after re-running the codesign commands the zip file is not identical.

I tried notarizing an existing signed Firefox Nightly dmg. We have a bunch of errors, but they fall into 3 buckets:

• The executable does not have the hardened runtime enabled. These pages describe how to fix this. It looks like bug 1470597 is hard-blocking the notarization work?
  • (I did download this binary from Dec 31; if we've landed the hardened runtime changes since then, I can retry with a newer binary.)
• The signature algorithm used is too weak. I'm not sure about this -- we're using current Developer IDs. It's possible our mac signing servers are using an older XCode or are configured to use an older algorithm; we'll have to dig.
• The binary is not signed. We have rules on the signing server about which binaries to skip; we can probably sign this binary as well.

(I know there's instructions above on how to sign the app before notarizing, but ideally we'd be able to keep our existing signing automation and just add the notarization piece. It looks like we have to at least adjust our signing for this to work.)

(In reply to Haik Aftandilian [:haik] from comment #4)

Removed the dependency on bug 1475652 (build with the 10.14 SDK). Using the
Notary Service does require opting into Enhanced Runtime, but Apple
confirmed enabling Enhanced Runtime doesn't require building with the 10.14
SDK. It does require running the codesign command on 10.13.6 or newer.

Doh. Our signing servers are 10.10.5.
We can either upgrade the mac signing server pool (or spin up a 2nd mac signing server pool and deprecate the original), or sign on the same macs that run the notarization steps.

I seem to recall hitting problems the last time we wanted to update which version of macOS the signing servers were running on. Something about compatibility for the signed binaries on older versions of macOS when they've been signed on newer versions of macOS.

According to https://www.mozilla.org/en-US/firefox/64.0.2/system-requirements/, we still support 10.9, so we should make sure that the new signed/notarized binaries still work properly on 10.9 systems.

From some brief research it sounds like if we sign on a newer version of macOS but continue to build with -mmacosx-version-min=10.9 we should be OK:

http://www.discretecosine.com/index-html-2/
https://stackoverflow.com/questions/41865537/how-does-apples-codesign-utility-decide-which-sha-algorithms-to-sign-a-shared

codesign will include both version 1 and 2 signatures if targeting macOS earlier than 10.10, and when targeting versions older than 10.12 codesign will include both sha-1 and sha-256 hashes in the code signature.

Using the Notary Service requires that the app not be codesigned with the com.apple.security.get-task-allow entitlement and this in turn prevents users from attaching a debugging to any of the browser processes. The workaround is to disable system integrity protection (SIP). I'm going to send a mail to dev-platform to get some feedback on this. If our developers want to continue to be able to debug builds on the Nightly channel (without the workaround) we will have to skip notarization on that channel.

Aki or Ted, if you're familiar with this, could you confirm when we run codesign today? For example, is it just on official channel builds and not for any local or try builds? Once we start using the notary service, the entitlements we use to codesign the binaries will affect execution in some ways. Initially, the only significant effect will be the ability to attach a debugger, but in the future we may want local/try builds to be codesigned with certain entitlements so those builds match what we ship to users (apart from allowing the debugger.)

We codesign in multiple places/branches, mainly for l10n-on-push iirc (l10n repacks repack signed binaries).

However, we have 3 levels of signing: release signing, nightly signing, and depend signing. (depend signing doesn't ship anywhere, and is only there for test/development purposes.) I believe the only difference today is which key they sign with, but I think we could add configuration to only notarize on release, or on release and nightly, as needed.

Attaching new screenshot comparing notarized first run dialog to not-notarized. When Apple announced this at WWDC the notarized first run dialog included the App icon, as shown in the earlier attached screenshot[1]. I'm not seeing that in my tests and will try to resolve this. If this has changed, then the earlier screenshot is now obsolete.

1. https://bugzilla.mozilla.org/attachment.cgi?id=8987232&action=edit

Image was edited to replace an internal IP address with "www.mozilla.org" on the notarized dialog.

Sounds like Aki answered your question. (FWIW I don't really know anything about our code signing process.)

Rob, I'll add you to this bug as Thunderbird will need this too.

Should we resolve this bug too?

Yes. Closing as a dupe of 1471004.