Summary:

Currently we don't have any telemetry measuring

• If a Firefox icon is "pinned" to the taskbar in the OS
• How a Firefox session was launched (pinned icon, desktop icon, start menu, or other)

We need that telemetry to exist (note, ideally we want this for windows, mac, and linux, but if it's significantly easier to just do so for windows, that's a choice we are okay with).

Why:

The UJET team will be investing significantly into pin-to-taskbar messaging in H1. We need this telemetry to understand how successful we are.

Also, this will be generally helpful, and lets us understand different use cases for our browser.

Timeline / Priority:

High. We would like this as soon as feasible.

References:

slack thread convo

We could detect how Firefox was launched with the help of GetStartupInfo, in the STARTUPINFO struct if the STARTF_TITLEISLINKNAME flag is set then lpTitle is the path to the shortcut. There's also STARTF_TITLEISAPPID, which might be involved when launching from a pin, I haven't checked. This information would need to be passed through the launcher process to the real parent process, but it should be feasible. (Thanks to mhowell for pointing this out!)

I added a pointer to bug 1243603, for reporting when launched with an external URL, as that would probably populate the same field in telemetry.

Any updates to timeline here?

We're preparing experiments relating to pinning that would benefit from this telemetry for measuring success. (perhaps even need it)

See:

• https://jira.mozilla.com/browse/UJET-72
• https://jira.mozilla.com/browse/UJET-75

Rachel, I'm flagging you here because I thought you would be the most likely to be able to answer the question in Comment 2.

I'm aiming to get this finished in time to ride with 87.

Adjusting priority here as another project is going to be bumped up before we can tackle this.

Here are my current plans for the feature, Romain (or others) please let me know if you have questions or would like me to change anything.

I'm planning to implement this for landing early on Nightly 88 give the change in priority.

My plan is to record an event near Firefox startup, to detect what shortcut was used to start. Checking whether we are pinned to the taskbar may require somewhat more I/O, as it may need to check every pin to determine whether it corresponds to this executable and AUMID, so I plan to defer reporting that to a best-effort, late idle task (as with installation telemetry and profile count) rather than include it in the same event, given that this can be expected to change during a run a histogram probably makes the most sense. As this part will be needed for bug 1686343 I will implement the check first in bug 1691455.

I don't plan to do any authentication on the shortcuts to determine whether they were made by our installer, just to check the location (Desktop, Start Menu, Taskbar, other). It would be possible to check for the exact file name that the installer would have used, but I don't think that will be sufficiently helpful given how rare user-created shortcuts are likely to be, and there could be complications around brand name migrations over the years. I could make an effort to tag our shortcuts in such a way as to distinguish them from user-crated shortcuts, but it also doesn't seem worth the complication.

Do we want to send an event on every startup, whether Firefox was started from a shortcut or not? This could provide space for future extension along the lines of bug 1243603, and would provide some baseline of all startups. But without being able to distinguish most of the other ways Firefox starts up, e.g. restarts for updates, OS restarts, I'm not sure how useful it would be. I will plan on only sending the event if we are started with a shortcut, but I'll leave the event name generic so it could be naturally extended to other startup methods.

(In reply to Adam Gashlin (he/him) [:agashlin] from comment #6)

Here are my current plans for the feature, Romain (or others) please let me know if you have questions or would like me to change anything.

I'm planning to implement this for landing early on Nightly 88 give the change in priority.

My plan is to record an event near Firefox startup, to detect what shortcut was used to start. Checking whether we are pinned to the taskbar may require somewhat more I/O, as it may need to check every pin to determine whether it corresponds to this executable and AUMID, so I plan to defer reporting that to a best-effort, late idle task (as with installation telemetry and profile count) rather than include it in the same event, given that this can be expected to change during a run a histogram probably makes the most sense. As this part will be needed for bug 1686343 I will implement the check first in bug 1691455.

This sounds fine to me, early collection of the data is not critical for this measurement.

I don't plan to do any authentication on the shortcuts to determine whether they were made by our installer, just to check the location (Desktop, Start Menu, Taskbar, other). It would be possible to check for the exact file name that the installer would have used, but I don't think that will be sufficiently helpful given how rare user-created shortcuts are likely to be, and there could be complications around brand name migrations over the years. I could make an effort to tag our shortcuts in such a way as to distinguish them from user-crated shortcuts, but it also doesn't seem worth the complication.
I'll defer to Alex here, unsure if the plan is to test efficiency of scenarios like telling users to right click on taskbar and create a shortcut VS us doing it on their behalf.
Also will we be able to segment out sessions that started from a launch of the executable directly as opposed to a shortcut?

Do we want to send an event on every startup, whether Firefox was started from a shortcut or not? This could provide space for future extension along the lines of bug 1243603, and would provide some baseline of all startups. But without being able to distinguish most of the other ways Firefox starts up, e.g. restarts for updates, OS restarts, I'm not sure how useful it would be. I will plan on only sending the event if we are started with a shortcut, but I'll leave the event name generic so it could be naturally extended to other startup methods.

I think it would be useful for at least scenarios where Firefox starts as a result of being default handler for a specific protocol or file extension (both I "think") are feasible but not part of the scope of this bug). The approach you suggest here would at least help quantify the 'other' bucket to help understand if we need to dig further on braking down what non shortcut initiated sessions are initiated from.
Note: installer telemetry tells us whether launching the executable from the stub was successful so perhaps this is a scenario that is technically addressable (telemetry to identify if Firefox was launched from the stub)

Hi Adam,

Thanks for the update. Thoughts below:

For the "is the browser pinned in taskbar or not" telemetry:

• I think the timing is fine, as long as it's early enough that we're confident we can reliability record this measurement on the vast majority of sessions.
• I'm not sure I understand the need to use a histogram here. You mentioned that this state (pinned, yes or no) is expected to change in a session, but I think it makes more sense to split this out into two separate measurements:
  • Measurement 1: is_pinned: (scalar), records once, and just tells us if it's pinned or not at the time of measurement
  • Measurement 2: pin_event: (event), fires off when there's a pinning event.

Splitting it out like this makes more sense for me from an analysis standpoint.

Also, some thoughts on the shortcut-startup measurement:

• I don't think authentication is necessary, location is fine
• leaving room for extension sounds good. it would be good to (eventually) have full coverage of how the session was initiated, something like:
  • shortcut:location
  • os_searchbar
  • thirdparty_link
  • command_prompt:flags
  • firstrun_installation
  • browser_restart
  • os_restart
  • unknown_other
• it sounds like you're planning to implement this as an event. any specific reason? it seems to me that having this as a scalar inside main pings makes sense. it would make analysis much more straightforward.

Thanks for the input!

Alex, there's a question here from Romain I'd like your input on:

(In reply to Romain Testard [:RT] from comment #7)

(In reply to Adam Gashlin (he/him) [:agashlin] from comment #6)

I don't plan to do any authentication on the shortcuts to determine whether they were made by our installer, just to check the location (Desktop, Start Menu, Taskbar, other). It would be possible to check for the exact file name that the installer would have used, but I don't think that will be sufficiently helpful given how rare user-created shortcuts are likely to be, and there could be complications around brand name migrations over the years. I could make an effort to tag our shortcuts in such a way as to distinguish them from user-crated shortcuts, but it also doesn't seem worth the complication.

I'll defer to Alex here, unsure if the plan is to test efficiency of scenarios like telling users to right click on taskbar and create a shortcut VS us doing it on their behalf.

Alex? I still think that just checking the location of a shortcut will get us 99% of the information we need.

Also will we be able to segment out sessions that started from a launch of the executable directly as opposed to a shortcut?

The values would be:

• start menu shortcut
• desktop shortcut
• taskbar shortcut
• other shortcut
• other

"other" covers a lot of ground (restarts, OS file type/protocol associations, etc.), but it does include launching the exe directly, and it does not include any kind of shortcut.

Do we want to send an event on every startup, whether Firefox was started from a shortcut or not? This could provide space for future extension along the lines of bug 1243603, and would provide some baseline of all startups. But without being able to distinguish most of the other ways Firefox starts up, e.g. restarts for updates, OS restarts, I'm not sure how useful it would be. I will plan on only sending the event if we are started with a shortcut, but I'll leave the event name generic so it could be naturally extended to other startup methods.

I think it would be useful for at least scenarios where Firefox starts as a result of being default handler for a specific protocol or file extension (both I "think") are feasible but not part of the scope of this bug).

Agreed, those are in principle possible, but let's leave them out for now.

The approach you suggest here would at least help quantify the 'other' bucket to help understand if we need to dig further on braking down what non shortcut initiated sessions are initiated from.
Note: installer telemetry tells us whether launching the executable from the stub was successful so perhaps this is a scenario that is technically addressable (telemetry to identify if Firefox was launched from the stub)

Ok, that makes sense. As Su-Young suggests below I'll use a scalar, but I'll still provide an "other" value so we can distinguish that from the scalar just not being set (e.g. after a subsession split).

(In reply to Su-Young Hong from comment #8)

Hi Adam,

Thanks for the update. Thoughts below:

For the "is the browser pinned in taskbar or not" telemetry:

• I'm not sure I understand the need to use a histogram here. You mentioned that this state (pinned, yes or no) is expected to change in a session, but I think it makes more sense to split this out into two separate measurements:
  • Measurement 1: is_pinned: (scalar), records once, and just tells us if it's pinned or not at the time of measurement
  • Measurement 2: pin_event: (event), fires off when there's a pinning event.

Splitting it out like this makes more sense for me from an analysis standpoint.

Ok, I can see the reasoning there.

Also, some thoughts on the shortcut-startup measurement:

• it sounds like you're planning to implement this as an event. any specific reason? it seems to me that having this as a scalar inside main pings makes sense. it would make analysis much more straightforward.

No great reason, I just added my first event so I've been somewhat in that frame of mind. If a scalar makes more sense I'll use that, thanks!

A point of clarification: For this telemetry and for the checks in bug 1687562, I'm considering "is Firefox pinned" as referring to the specific running installation, ignoring other Firefox installations on the system. This is the simplest to implement, and it doesn't break in the multi-Firefox case.

[Tracking Requested - why for this release]: Flagging this one early for uplift as we'd like to try to get it into FX 87 if at all possible. Understandable that can't be approved until the patch is ready, but I'm flagging here to give as much advance warning as we have.

(In reply to Adam Gashlin (he/him) [:agashlin] from comment #9)

Thanks for the input!

Alex, there's a question here from Romain I'd like your input on:

(In reply to Romain Testard [:RT] from comment #7)

(In reply to Adam Gashlin (he/him) [:agashlin] from comment #6)

I don't plan to do any authentication on the shortcuts to determine whether they were made by our installer, just to check the location (Desktop, Start Menu, Taskbar, other). It would be possible to check for the exact file name that the installer would have used, but I don't think that will be sufficiently helpful given how rare user-created shortcuts are likely to be, and there could be complications around brand name migrations over the years. I could make an effort to tag our shortcuts in such a way as to distinguish them from user-crated shortcuts, but it also doesn't seem worth the complication.

I'll defer to Alex here, unsure if the plan is to test efficiency of scenarios like telling users to right click on taskbar and create a shortcut VS us doing it on their behalf.

Alex? I still think that just checking the location of a shortcut will get us 99% of the information we need.

If I understand correctly, you're wondering if we want to know how a shortcut was created or if there just is one created regardless if it was user-generated or browser-generated.
The most important is that we can detect if there's a pinned shortcut. How it got there is a bonus but not necessary if it adds a lot of scope.

Also will we be able to segment out sessions that started from a launch of the executable directly as opposed to a shortcut?

The values would be:

• start menu shortcut
• desktop shortcut
• taskbar shortcut
• other shortcut
• other

"other" covers a lot of ground (restarts, OS file type/protocol associations, etc.), but it does include launching the exe directly, and it does not include any kind of shortcut.

We don't support Tiles yet in Win 10 but I could see that being an option in the near-medium term. Not sure if we already have to support it for that to be a potential value.

And perhaps, you're bundling the Tile with the "start menu shortcut" which could work but we'd lose some granularity.

(In reply to Alex Davis [:adavis] from comment #12)

We don't support Tiles yet in Win 10 but I could see that being an option in the near-medium term. Not sure if we already have to support it for that to be a potential value.

And perhaps, you're bundling the Tile with the "start menu shortcut" which could work but we'd lose some granularity.

They're bundled, I don't know a way to distinguish launching from a pinned start menu shortcut (which I think is the Tile meant here?) vs. any other start menu shortcut, they use the same .lnk file. However, if we're eventually creating the start menu pin from Firefox or the installer (I don't know how this works), that might look different.

(In reply to Adam Gashlin (he/him) [:agashlin] from comment #13)

They're bundled, I don't know a way to distinguish launching from a pinned start menu shortcut (which I think is the Tile meant here?) vs. any other start menu shortcut, they use the same .lnk file.

It turns out that on Windows 7 there was a specific folder where Start Menu pins go, similar to Taskbar pins. Rather than split this out, I've chosen to bundle it with the other Start Menu shortcuts, for consistency between Windows 10 and Windows 7. I could just as easily split it out, but it doesn't seem worth it for something that would only allow analysis of Windows 7, as far as I can tell.

Depends on D106343

Depends on D106344

Depends on D106345

Depends on D106346

Comment on attachment 9205271 [details]
launch telemetry data review request

lgtm, ty!

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, in the probe definition files and the Probe Dictionary.

2. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, the Firefox telemetry opt-out.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

n/a

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Cat 2, interaction data

5. Is the data collection request for default-on or default-off?

Default-on

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No

7. Is the data collection covered by the existing Firefox privacy notice?

Yes

8. Does there need to be a check-in in the future to determine whether to renew the data?

No

9. Does the data collection use a third-party collection tool?

No

https://hg.mozilla.org/mozilla-central/rev/c2e2d914dc3d
https://hg.mozilla.org/mozilla-central/rev/32f36b39c905
https://hg.mozilla.org/mozilla-central/rev/a3cc46b7462b
https://hg.mozilla.org/mozilla-central/rev/bf1395eaaef2
https://hg.mozilla.org/mozilla-central/rev/a6dfee675aa9

Comment on attachment 9205242 [details]
Bug 1685213 - Part 1: Save and access the startup shortcut. r?bytesized!

Beta/Release Uplift Approval Request

• User impact if declined: We are starting experiments to test functionality around taskbar pinning in the onboarding flow (see 1686343 with FX 87. Without these additional telemetry changes, we will not be able to accurately measure the impact of those experiments. For a user, this means we could risk displaying a non-winning variation for longer than we might otherwise.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): It's a telemetry change only, which is why we have deemed this low-risk.
• String changes made/needed:

Comment on attachment 9205242 [details]
Bug 1685213 - Part 1: Save and access the startup shortcut. r?bytesized!

approved for 87.0b5

I applied and built the patch series on top of 87.0b4, it passes some cursory tests. Preliminary results from Nightly (internal STMO link) look correct.

I'm ok with waiting until a few more days of Nightly, or ideally until QA is able to run it through the paces, before uplifting. But early results are as expected and as Rachel says the risk is fairly low.

I have verified this issue using the latest Firefox Nightly 88.0a1 (Build ID: 20210301220015) on Windows 10 x64 and Windows 8.1 x64. The scenarios tested are described as follow:

• The “os.environment.is_taskbar_pinned” scalar is registered on the “about:telemetry#scalars-tab" page and has the “false” value when the Firefox browser is not pinned to Taskbar and is launched through:
  • Firefox Desktop shortcut;
  • Firefox Startmenu shortcut;
  • Mailto protocol;
  • HTML extension File;
  • Profile manager;
  • Command line.
• The “os.environment.is_taskbar_pinned” scalar is registered on the “about:telemetry#scalars-tab" page and has the “true” value when the Firefox browser is pinned to Taskbar and is launched through:
  • Firefox Desktop shortcut;
  • Firefox Startmenu shortcut;
  • Firefox Taskbar shortcut;
  • Mailto protocol;
  • HTML extension File;
  • Profile manager;
  • Command line.
• The “os.environment.launch_method” scalar is registered on the “about:telemetry#scalars-tab" page and has the “Desktop” value when the Firefox browser is launched through:
  • Firefox Desktop shortcut;
  • Firefox Desktop shortcut copied in another folder;
  • Command line.
• The “os.environment.launch_method” scalar is registered on the “about:telemetry#scalars-tab" page and has the “StartMenu” value when the Firefox browser is launched through:
  • Firefox Startmenu shortcut;
• The “os.environment.launch_method” scalar is registered on the “about:telemetry#scalars-tab" page and has the “Taskbar” value when the Firefox browser is launched through:
  • Firefox Taskbar shortcut;
• The “os.environment.launch_method” scalar is registered on the “about:telemetry#scalars-tab" page and has the “Other” value when the Firefox browser is launched through:
  • Mailto protocol;
  • HTML extension File;
  • Profile manager;

Also, we will perform a more thoughtfully testing round in the Firefox 87 Pre-Release phase which will conclude with a testing report.

https://hg.mozilla.org/releases/mozilla-beta/rev/91ca92164ab1
https://hg.mozilla.org/releases/mozilla-beta/rev/67df42acb76e
https://hg.mozilla.org/releases/mozilla-beta/rev/618f799f928d
https://hg.mozilla.org/releases/mozilla-beta/rev/f8aad995e7dd
https://hg.mozilla.org/releases/mozilla-beta/rev/3cd359e37116

We’ve finished running the feature Test Cases and the “os.environment.is_taskbar_pinned” and “os.environment.launch_method” scalars are correctly registered in the browser. We’ve verified this issue using the Firefox Beta 87.0b5 (Build ID: 20210302185821) on Windows 10 x64, Windows 8.1 x64 and Windows 7 x64.