In bug 1496380 I added an API on the external protocol service that takes an arbitrary protocol string and returns whether or not Firefox is the default.

It looks like the shell service on mac and windows does things slightly differently - on Windows, it does some strict checks on the exact values of the registry entry (unsure why?), and on mac it cares about the app bundle identifier rather than the actual .app file (so if some other Firefox is the default then it'll say "we're the default browser", even if the specific running app is not).

At least on Windows, it may be that we also want the mime service to provide a similar API for files so we can check if we're the default app for HTML, too.