Firefox launched on the MS store and we see new profiles with distribution_id='mozilla-MSIX' although we're unclear on the share of new MSIX profiles coming from Windows systems where Firefox was already installed.
This would be useful in order to understand the overall value of the MS Store acquisition channel.

Acceptance criteria:

• Identify if at the time of the MSIX installation or new profile creation another Firefox installation was already present on the Windows system and place that piece of information on the telemetry information that gets sent with the new profile ping

Nick, I've been sketching this out a bit. Can you make sure I'm on the right path before I dive too deep?

It sounds like we want to add to the installation telemetry ping for this. I'm thinking about a new key in extra_keys called any_install_existed that would be true or false, depending on whether or not we found an existing non-MSIX Firefox install. If we care to look for existing MSIX installs as well, we can add an additional existing_msix_install_existed for that.

As far as how to detect the installs, it seems like the registery is our best bet. {HKCU,HKLM}\Software\Mozilla appears to contain a key for every install (the ones with a name + version). If any of those are present, I think we can be reasonably certain that an install exists. And if we want to be absolutely certain, checking the path of bin\PathToExe seems like it can provide that.

If we're going to check for existing MSIX packages (Store or otherwise), it looks like https://docs.microsoft.com/en-us/uwp/api/windows.management.deployment.packagemanager.findpackage?view=winrt-22000 or https://docs.microsoft.com/en-us/uwp/api/windows.management.deployment.packagemanager.findpackages?view=winrt-22000 should make this pretty easy.

(In reply to bhearsum@mozilla.com (:bhearsum) from comment #1)

Nick, I've been sketching this out a bit. Can you make sure I'm on the right path before I dive too deep?

'course!

It sounds like we want to add to the installation telemetry ping for this. I'm thinking about a new key in extra_keys called any_install_existed that would be true or false, depending on whether or not we found an existing non-MSIX Firefox install. If we care to look for existing MSIX installs as well, we can add an additional existing_msix_install_existed for that.

So, nomenclature first: I do not think that this is a separate "installation telemetry ping". I think it's a telemetry event that happens to be sent at first run.

As far as how to detect the installs, it seems like the registery is our best bet. {HKCU,HKLM}\Software\Mozilla appears to contain a key for every install (the ones with a name + version). If any of those are present, I think we can be reasonably certain that an install exists. And if we want to be absolutely certain, checking the path of bin\PathToExe seems like it can provide that.

We have code doing exactly this that is ripe for re-use and generalization in the uninstall ping: https://searchfox.org/mozilla-central/source/toolkit/components/telemetry/pings/UninstallPing.jsm. I don't know why we choose to only put this valuable data in the uninstall ping, but here we are. I'd like to consider making it more available, i.e., in more pings. (Especially 'cuz uninstall pings are not guaranteed to happen at all!) It feels like it should just be part of the telemetry environment, but it's rather expensive to check, and I believe that the environment is considered "sealed" and shouldn't be a dumping ground for such things. Perhaps just regular old main ping telemetry, produced lazily and/or cached on disk (in the profile, perhaps)?

If we're going to check for existing MSIX packages (Store or otherwise), it looks like https://docs.microsoft.com/en-us/uwp/api/windows.management.deployment.packagemanager.findpackage?view=winrt-22000 or https://docs.microsoft.com/en-us/uwp/api/windows.management.deployment.packagemanager.findpackages?view=winrt-22000 should make this pretty easy.

Yes, I would like to extend this to search for installed packages, filtering to just the names that we ourselves produce: maybe Mozilla.Firefox* and Mozilla.MozillaFirefox* to start? And yes, these APIs are the ones I would use to fish this information.

In conclusion: I don't feel too strongly about the single installation event vs. generally available. Romain can decide how "live" this needs to be. I would like to see us lift the code from the uninstall ping for re-use -- you might start by looking at https://phabricator.services.mozilla.com/D108662 and its comments re: performance -- and to add new code that uses the MS APIs to fish installed packages.

If we stick that into the installation event (and into the uninstall ping?), that might do the trick. Let me know if anything seems off here.

Thank you for all of this! I think it's more than enough to move forward.

(In reply to Nick Alexander :nalexander [he/him] from comment #2)

(In reply to bhearsum@mozilla.com (:bhearsum) from comment #

As far as how to detect the installs, it seems like the registery is our best bet. {HKCU,HKLM}\Software\Mozilla appears to contain a key for every install (the ones with a name + version). If any of those are present, I think we can be reasonably certain that an install exists. And if we want to be absolutely certain, checking the path of bin\PathToExe seems like it can provide that.

We have code doing exactly this that is ripe for re-use and generalization in the uninstall ping: https://searchfox.org/mozilla-central/source/toolkit/components/telemetry/pings/UninstallPing.jsm. I don't know why we choose to only put this valuable data in the uninstall ping, but here we are. I'd like to consider making it more available, i.e., in more pings. (Especially 'cuz uninstall pings are not guaranteed to happen at all!) It feels like it should just be part of the telemetry environment, but it's rather expensive to check, and I believe that the environment is considered "sealed" and shouldn't be a dumping ground for such things. Perhaps just regular old main ping telemetry, produced lazily and/or cached on disk (in the profile, perhaps)?

If we're sending it for more than just the installation event, it does seem like we need to address the perf considerations. If we're caching, presumably we need a cheap way to invalidate the cache (perhaps every N days?).I'll give this some more thought if we end up going the more available route.

In conclusion: I don't feel too strongly about the single installation event vs. generally available. Romain can decide how "live" this needs to be.

Romain, ni? for you on this part.

Using the installation event feels right to me - I current analyze installs through different angles and I could see a similar approach to the following query to analyze MSIX installations where Firefox was already installed: https://sql.telemetry.mozilla.org/queries/80491#199657

One thing I noticed on installation.first_seen is that we collect the "from_msi" field. This was done to segment msi and exe installs but we now have the new msix type and for consistency and analysis simplicity (the distribution_id is not available on all tables) perhaps we should also add a from_msix field?
In any case a existing_msix_install_existed field similar to any_install_existed makes sense to me - I assume that if a system accounts for 1 exe installs and 1 msix install then existing_msix_install_existed and any_install_existed will both be true?

(In reply to Romain Testard [:RT] from comment #5)

One thing I noticed on installation.first_seen is that we collect the "from_msi" field. This was done to segment msi and exe installs but we now have the new msix type and for consistency and analysis simplicity (the distribution_id is not available on all tables) perhaps we should also add a from_msix field?

I don't see any reason we can't add this as well, if it's useful.

In any case a existing_msix_install_existed field similar to any_install_existed makes sense to me - I assume that if a system accounts for 1 exe installs and 1 msix install then existing_msix_install_existed and any_install_existed will both be true?

Yup, that's right.

Nick, I'm needinfo'ing you as requested re: the perf considerations of https://phabricator.services.mozilla.com/D108662. I think you explained it well enough on Slack in that it's removing the limit of installs to look for, which means we'll spend more time in any case where there's more than 11 installs present.

I'd also love to get some high level feedback on https://phabricator.services.mozilla.com/D134326. The guts and tests need a lot of fleshing out still, but it would be good to know if the skeleton is in decent shape.

(In reply to bhearsum@mozilla.com (:bhearsum) from comment #9)

Nick, I'm needinfo'ing you as requested re: the perf considerations of https://phabricator.services.mozilla.com/D108662. I think you explained it well enough on Slack in that it's removing the limit of installs to look for, which means we'll spend more time in any case where there's more than 11 installs present.

Correct. I think we need to fold the maximum check into the iteration, and then make the maximum a parameter to cater for the perf-sensitive use case (startup) and the less sensitive use cases (this telemetry?). If I were doing the mechanics of that myself, I'd start with my extraction patch and then absorb some changes to limit the recursion based on the parameter, but do as you see fit.

I'd also love to get some high level feedback on https://phabricator.services.mozilla.com/D134326. The guts and tests need a lot of fleshing out still, but it would be good to know if the skeleton is in decent shape.

Sorry that this slipped through the cracks: I had several working days before Christmas wiped out by yet another cold. I'll look now.

I'm not sure this is worthwhile keeping at this point, as we can't track "reinstalls" with MSIX the way we can other installers (upgrades and any other type of install all change the InstalledDate for an MSIX package). However, it's written now so I thought I'd put it up at least, and see what folks think.

Depends on D133747

Depends on D135245

Romain, this is still going through review but I'd like to double check the new information going into the installation event to be certain it aligns with your expectations. There's three new pieces of data being added:

• event_object will be set to msix for MSIX installations -- which means we don't need from_msix (and we're at our limit for fields anyways, so I don't think we can add it...)
• other_inst will be true if any non-MSIX install other than the currently running one was present when the ping was prepared
• other_msix_inst will be true if any MSIX install other than the currently running one was present when the ping was prepared

Effectively, this means that regardless of installation method, one or both of the above fields will be true if any other install is present on the system.

Thanks for checking in, overall this looks good to me, one clarifying questions below:
(In reply to bhearsum@mozilla.com (:bhearsum) from comment #13)

Romain, this is still going through review but I'd like to double check the new information going into the installation event to be certain it aligns with your expectations. There's three new pieces of data being added:

• event_object will be set to msix for MSIX installations -- which means we don't need from_msix (and we're at our limit for fields anyways, so I don't think we can add it...)
  will this only apply to msix or will this be set to exe and msi where applicable also? In that case from_msi is no more relevant?

(In reply to Romain Testard [:RT] from comment #14)

Thanks for checking in, overall this looks good to me, one clarifying questions below:
(In reply to bhearsum@mozilla.com (:bhearsum) from comment #13)

Romain, this is still going through review but I'd like to double check the new information going into the installation event to be certain it aligns with your expectations. There's three new pieces of data being added:

• event_object will be set to msix for MSIX installations -- which means we don't need from_msix (and we're at our limit for fields anyways, so I don't think we can add it...)

will this only apply to msix or will this be set to exe and msi where applicable also? In that case from_msi is no more relevant?

Only to MSIX. The other possible values are "stub" and "full", so is_msi is still needed to distinguish those installs.

I assume this did not make it to 97 Beta, how feasible is an uplift? We need this data to make Store investment decisions this year.

(In reply to Romain Testard [:RT] from comment #16)

I assume this did not make it to 97 Beta, how feasible is an uplift? We need this data to make Store investment decisions this year.

That's right, sorry about that. It's nearly ready to go, and I think it's safe enough to uplift. I'll talk to RelMan about it.

Comment on attachment 9258496 [details]
data review request

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Ben Hearsum is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does the data collection use a third-party collection tool?

No.

Result: datareview+

Are we still OK to uplift this?

(In reply to Romain Testard [:RT] from comment #21)

Are we still OK to uplift this?

I think so. If the latest version of the patches is OK I'll get it landed on Nightly today, and request the uplift tomorrow.

QA, the main thing to verify here is to ensure that full installer installations and MSIX installations accurately find (or don't find) other installs of both of those types. I would suggest using this instructions below to do so, starting from a system with no existing Firefox installations. Testing on Windows 7, 8, 10, and 11 would be ideal -- but 10 & 11 are the most important ones.

This hasn't yet landed on mozilla-central, so I would suggest waiting until it has to do the testing.

You can find the full and MSIX installers needed in https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/, and should include these patches by tomorrow.

Full installer, no existing installations

1. Install with the Full Installer and run Firefox
2. Set devtools.chrome.enabled to True
3. Open the Browser Console
4. Type BrowserUsageTelemetry.reportInstallationTelemetry()
5. Go to about:telemetry#events-tab
6. Search for install

There should be one installation event shown with object set to full and other_inst and other_msix_inst set to false

Full installer, existing full installer installation

Same instructions as above, but install a second time to a custom directory (so you have two installs).

There should be one installation event shown with object set to full and other_inst set to true and other_msix_inst set to false

Full installer, existing msix installation

Same instructions as above, but also install Firefox through the Microsoft Store before running Firefox.

There should be one installation event shown with object set to full and other_inst set to false and other_msix_inst set to true

MSIX installation, no existing installs

1. Install an official MSIX build and run Firefox
2. Set devtools.chrome.enabled to True
3. Open the Browser Console
4. Type BrowserUsageTelemetry.reportInstallationTelemetry()
5. Go to about:telemetry#events-tab
6. Search for install

There should be one installation event shown with object set to msix and other_inst and other_msix_inst set to false

MSIX installation, existing full installer installation

Same instructions as above, but also install with the full installer before running Firefox.

There should be one installation event shown with object set to msix and other_inst set to true and other_msix_inst set to false

MSIX installation, existing msix installation

Same instructions as above, but also install Firefox through the Microsoft Store before running Firefoex.

There should be one installation event shown with object set to msix and other_inst set to false and other_msix_inst set to true

https://hg.mozilla.org/mozilla-central/rev/ffb3a65c65ae
https://hg.mozilla.org/mozilla-central/rev/6dcac3b6ea09
https://hg.mozilla.org/mozilla-central/rev/79097ddba37b
https://hg.mozilla.org/mozilla-central/rev/529175563fd6

This still needs QA on it, but we are getting data on Nightly now. Here's a quick and dirty query with counts by installer type: https://sql.telemetry.mozilla.org/queries/83838/source

(In reply to bhearsum@mozilla.com (:bhearsum) from comment #26)

This still needs QA on it, but we are getting data on Nightly now. Here's a quick and dirty query with counts by installer type: https://sql.telemetry.mozilla.org/queries/83838/source

Ack, I just noticed that I forgot to save my query yesterday. This has now been updated to actually show what it should - and it seems pretty sane with 2 days of data.

Verified the scenarios from Comment 24 with Firefox 98 under Windows 11, 10 and 7. No issues were encountered.

Comment on attachment 9255248 [details]
Bug 1743465: add XPCOM component for interacting with Windows PackageManager r?mhowell!

Beta/Release Uplift Approval Request

• User impact if declined: No direct user impact
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: It would be good to test this on Beta, if/when it lands there. Particularly through the Microsoft Store, which is a test we couldn't do on Nightly.
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Because:
• Most of this code is covered by automated tests (except the MSIX-specific paths)
• The only impact to any errors would be seen in Telemetry (ie: not user facing)
• String changes made/needed: No

Comment on attachment 9255248 [details]
Bug 1743465: add XPCOM component for interacting with Windows PackageManager r?mhowell!

Makes me a bit nervous taking this in the final beta of the cycle, but the risk seems contained at least. Approved for 97.0b9 and I'll make sure the builds get uploaded to the MSFT Store also for the benefit of QA validation.

Comment on attachment 9256240 [details]
Bug 1743465: send additional information about existing installations in installation.first_seen Telemetry event. r?nalexander!

This needs rebasing around bug 1747461 as that doesn't look like something we can trivially uplift due to other dependencies. Please attach a revised patch for Beta ASAP.

This should apply cleanly to Beta. It was a simple rebase, so I don't anticipate any issues. Nevertheless, I have a try push going with it over at https://treeherder.mozilla.org/jobs?repo=try&revision=54938c1232fc53a66ec2ec45883a5a904a4a049f, but it's going to take awhile to complete the Windows 10 tests due to backlogs.

https://hg.mozilla.org/releases/mozilla-beta/rev/0d195331dbc3
https://hg.mozilla.org/releases/mozilla-beta/rev/88af816e7914
https://hg.mozilla.org/releases/mozilla-beta/rev/036009915799
https://hg.mozilla.org/releases/mozilla-beta/rev/af2f45bef3d1

Verified the scenarios with the beta MSIX Store and full installer from archive (both 97.0b9) on two different systems under Windows 11, 10 and 7. The "other_inst" value is always true (tried deleting registries, folder with profiles, etc), other than that, there were no other issues encountered.

Is the "other_inst=true" a reason to worry about?

(In reply to Catalin Sasca, QA [:csasca] from comment #34)

Verified the scenarios with the beta MSIX Store and full installer from archive (both 97.0b9) on two different systems under Windows 11, 10 and 7. The "other_inst" value is always true (tried deleting registries, folder with profiles, etc), other than that, there were no other issues encountered.

Is the "other_inst=true" a reason to worry about?

This is surprising to me, especially because we now have real world data with it false. There's a handful of places where you could have old registry entries causing this. They are:

• HKLM\Software\Mozilla\Firefox\TaskBarIDs
• HKLM\Software\Wow6432Node\Mozilla\Firefox\TaskBarIDs
• HKCU\Software\Mozilla\Firefox\TaskBarIDs
• HKCU\Software\Wow6432Node\Mozilla\Firefox\TaskBarIDs

If these keys are all empty and you're still experiencing this, that would be a bit concerning.

Seems like there was one in HKLM\Software\Wow6432Node\Mozilla\Firefox\TaskBarIDs which we didn't previously delete. After deleting this one, it did the trick and "other_inst" was false.