When the add-on signing requirement is turned on all our test harnesses which currently install as extensions will need to be signed in order to work in release and beta builds.

Dave, I assume that this not only applies to the harness itself but also to any XPI files as used by tests? That would be a lot of work I assume.

(In reply to Henrik Skupin (:whimboo) from comment #1) > Dave, I assume that this not only applies to the harness itself but also to > any XPI files as used by tests? That would be a lot of work I assume. That's right. I've just filed bug 1133838 to track the browser-chrome and xpcshell tests in toolkit/mozapps/extensions specifically but I guess there are others elsewhere and the solution may be different for each.

This is likely primarily to be a releng task.

Looks like basically all the test harnesses fail when signing is enabled: https://treeherder.mozilla.org/#/jobs?repo=try&revision=c56931f79aea

Kim, is this something you're working on?

One thing we have to be careful of here is that the signed harness extensions aren't something that someone can take and use to inject code into Firefox

(In reply to Dave Townsend [:mossop] from comment #6) > One thing we have to be careful of here is that the signed harness > extensions aren't something that someone can take and use to inject code > into Firefox Do you have suggestions on how to accomplish that?

The plan is to have signed extensions uplifted as part of the marge process but I have not been actively working on this, finishing up work on bug 1135781 first, then will move on to tests.

(In reply to Jonathan Griffin (:jgriffin) from comment #7) > (In reply to Dave Townsend [:mossop] from comment #6) > > One thing we have to be careful of here is that the signed harness > > extensions aren't something that someone can take and use to inject code > > into Firefox > > Do you have suggestions on how to accomplish that? If the tests are being included in the signed extension then that is probably enough. I'd be concerned if we're signing the test harness and then it was loading code to run from outside the signed extension.

Tests are generally not included in extensions atm; packaging the harnesses as extensions complete with tests would be a significant change from current procedure, and would make updating tests on beta/release very difficult, unless the test extension signing was part of the normal build process. This isn't likely achievable by the next uplift in Sept; what we should probably target is having the tests run at all using signed addons, and then look at hardening our approach for subsequent releases. I'll set up a meeting next week so we can discuss this.

(In reply to Jonathan Griffin (:jgriffin) from comment #10) > Tests are generally not included in extensions atm; packaging the harnesses > as extensions complete with tests would be a significant change from current > procedure, and would make updating tests on beta/release very difficult, > unless the test extension signing was part of the normal build process. > > This isn't likely achievable by the next uplift in Sept; what we should > probably target is having the tests run at all using signed addons, and then > look at hardening our approach for subsequent releases. Ok. A couple of things we could do to mitigate this would be to make sure we give the harness extension a specific version for each Firefox version, that way we can blocklist older versions if we become concerned. Also marking the min/maxVersions for the extension to match the Firefox version they are built for and include the strictCompatibility flag so they can't be used on other versions.

We had a meeting yesterday on this issue, here are the notes https://etherpad.mozilla.org/addon-signing-continous-integration-strategy is summary Kev to talk with mossop re enabling dev mode in Firefox which would allow us to test the addons without the need for implementing another build type or massive refactoring of test harnesses

Can this be closed now?