[Affected versions]:

• Firefox Nightly 81.0a1 - Build ID: 20200823211533

[Affected Platforms]:

• Windows 10 x64
• Windows 7 x64

[Prerequisites]:

• Have the "browser.ping-centre.log" pref set to "true" in the "about:config" page.
• Have the "Browser Console" opened.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Open a new tab and click any "Top Site" card.
3. Focus the "Browser Console" and observe the displayed telemetry pings.

[Expected result]:

• An "activity-stream" telemetry ping is displayed.

[Actual result]:

• There are no telemetry pings displayed at all.

[Regression Window]:

• Considering the fact that this issue is not reproducible on older Nightly versions, using the mozregression tool I have managed to find the following regression window:
  Last good revision: c0a323ca7d8b71128945fbff73c92444d8f7a424
  First bad revision: 751c328cd945915d48392ee7017357682bf8ce46
  Pushlog:
  https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=c0a323ca7d8b71128945fbff73c92444d8f7a424&tochange=751c328cd945915d48392ee7017357682bf8ce46

From the pushlog it seems that Bug 1642295 has caused this behavior.

[Notes]:

• This issue is not reproducible on macOS and Linux.
• The "Structured Ingestion ping failure with error: error" error is displayed in the "Browser Console".
• I have verified using a local telemetry server that the telemetry pings are generated and successfully send to the server.
• Attached a screen recording of the issue.

@Julian, could you please take a look over this issue?

I'm going to look at this.

Do these pings work on local builds (i.e. when building from mozilla-central. i.e. ./mach build && ./mach run with artifact builds)?
I'm not able to see them, even without Julian changes.
I tested on https://hg.mozilla.org/mozilla-central/rev/f749582822e8

After setting:

• browser.newtabpage.activity-stream.telemetry
• browser.newtabpage.activity-stream.telemetry.ut.events
• browser.ping-centre.log

to true, I'm able to see logs in the browser console

TELEMETRY PING (activity-stream): {"experiments":{},"locale":"en-US","version":"81.0a1","release_channel":"default","message_id":"20745","source":"NEWTAB_FOOTER_BAR","event":"IMPRESSION","id":"NEWTAB_FOOTER_BAR","addon_version":"20200824094458","client_id":"d1dc31b9-88c3-4535-a64e-ebd9ba75984f"}

TELEMETRY PING (activity-stream): {"experiments":{},"locale":"en-US","version":"81.0a1","release_channel":"default","message_id":"FXA_ACCOUNTS_BADGE","bucket_id":"FXA_ACCOUNTS_BADGE","event":"IMPRESSION","addon_version":"20200824094458","client_id":"d1dc31b9-88c3-4535-a64e-ebd9ba75984f","browser_session_id":"cb084045-9c7c-4fa4-a75d-605562098025"}

TELEMETRY PING (activity-stream): {"experiments":{},"locale":"en-US","version":"81.0a1","release_channel":"default","message_id":"20267","source":"NEWTAB_FOOTER_BAR","event":"IMPRESSION","id":"NEWTAB_FOOTER_BAR","addon_version":"20200824094458","client_id":"d1dc31b9-88c3-4535-a64e-ebd9ba75984f"}

TELEMETRY PING (activity-stream): {"experiments":{},"locale":"en-US","version":"81.0a1","release_channel":"default","addon_version":"20200824094458","user_prefs":255,"session_id":"{b99e1a80-f50b-4dc1-b683-5a02ddf96897}","page":"about:newtab","session_duration":4260,"perf":{"load_trigger_type":"unexpected","is_preloaded":true,"topsites_icon_stats":{"custom_screenshot":0,"screenshot_with_icon":0,"screenshot":0,"tippytop":4,"rich_icon":2,"no_image":0},"topsites_pinned":1,"topsites_search_shortcuts":1,"topsites_first_painted_ts":1598279033835,"visibility_event_rcvd_ts":1598279048442,"load_trigger_ts":-1},"profile_creation_date":18385,"client_id":"d1dc31b9-88c3-4535-a64e-ebd9ba75984f"}

Hi Nicolas, I have verified and I can confirm that the browser.newtabpage.activity-stream.telemetry, browser.newtabpage.activity-stream.telemetry.ut.events, and thebrowser.ping-centre.logprefs are set totrue` in the "about:config" page, however, the telemetry pings are still not displayed inside the "Browser Console".

Tested using the latest Firefox Nightly (en-US locale build) 82.0a1 - Build ID: 20200824215021 on Windows 10 x64.

(In reply to Marius Coman [:mcoman], Ecosystem QA from comment #4)

Hi Nicolas, I have verified and I can confirm that the browser.newtabpage.activity-stream.telemetry, browser.newtabpage.activity-stream.telemetry.ut.events, and thebrowser.ping-centre.logprefs are set totrue` in the "about:config" page, however, the telemetry pings are still not displayed inside the "Browser Console".

Tested using the latest Firefox Nightly (en-US locale build) 82.0a1 - Build ID: 20200824215021 on Windows 10 x64.

Marius, could you check the values of the following pref in about:config please:

• fission.autostart
• devtools.browsertoolbox.fission
• devtools.contenttoolbox.fission
• devtools.target-switching.enabled
• devtools.testing.enableServerWatcherSupport

The following pref values are present in the "about:config" page:

• fission.autostart is set to false
• devtools.browsertoolbox.fission is set to true
• devtools.contenttoolbox.fission is set to true
• devtools.target-switching.enabled is set to false
• devtools.testing.enableServerWatcherSupport is not set in the "about:config" page.

Hope it helps :)

With the exact same preferences, I can see the pings 🤔
Could you put a dump call just before https://searchfox.org/mozilla-central/rev/73a14f1b367948faa571ed2fe5d7eb29460787c1/browser/modules/PingCentre.jsm#196-198 to be extra sure we do enter this if block?

dump(`TELEMETRY PING (${this._topic}): ${payload}\n`)

Hey Nicolas,

I’ve discussed with Marius and we’ve investigated this issue a bit further and found out that it seems to be caused by an environment variable that we have to set in order to be able to enroll in the recipes created on the Stage Normandy server. What we are doing is the following:

• On Windows we set a System Variable named XPCSHELL_TEST_PROFILE_DIR with the value set to foo.
• On macOS, and Linux we have to write the following line in the terminal export XPCSHELL_TEST_PROFILE_DIR=foo and then start the Firefox browser from the same terminal.
  Without this variable set, the Telemetry events appear in the Browser Console as expected.

@Nicolas, could you please set the same variable as us and see if the issue is also reproducible on your end?

I’ve also observed that in the latest Firefox Nightly 82.0a1 (Build ID: 20200826214121) and the latest Firefox Beta 81.0b2 (Build ID: 20200825191644) the Executing recipe <example> … messages do not appear in the Browser Console. I’ve run a regression window for this particular behavior and reached the same bug as Marius in comment 0. Based on this I’m unsure what exactly has caused this.

@mythmon, @Mathew, could you please take a look at the issue I’ve described in the 2nd part of this comment? Should this behavior be tracked separately in a different component (Normandy Client/Remote Settings Client)?

I expect that the lack of "Executing recipe" logs is caused by the same thing as the lack of Telemetry Pings in the console. I don't think this needs a separate bug, and if it does I don't think it would be in the Normandy Client or Remote Settings Client components.

Might be related, I noticed when bug 1642295 landed, I no longer see logs at all in the browser console, you can test this with a simple console.log if the multi processes debugger is open.

(In reply to Robert Martin [:romartin], Ecosystem QA from comment #8)

Hey Nicolas,

I’ve discussed with Marius and we’ve investigated this issue a bit further and found out that it seems to be caused by an environment variable that we have to set in order to be able to enroll in the recipes created on the Stage Normandy server. What we are doing is the following:

• On Windows we set a System Variable named XPCSHELL_TEST_PROFILE_DIR with the value set to foo.
• On macOS, and Linux we have to write the following line in the terminal export XPCSHELL_TEST_PROFILE_DIR=foo and then start the Firefox browser from the same terminal.
  Without this variable set, the Telemetry events appear in the Browser Console as expected.

@Nicolas, could you please set the same variable as us and see if the issue is also reproducible on your end?

yes, with the system variable (and after a restart), I'm able to see the issue.
In DevTools we have a check on this variable to handle debugging xpcshell tests: devtools/server/actors/descriptors/process.js#64-72
When we see that the variable exists, we don't create the same target actor (we create a content process target instead of a parent process target), which I guess then impacts how we listen for things.

Are there any reason to set this specific system variable?
This is used in a few places to check that we're running an xpcshell test (https://searchfox.org/mozilla-central/search?q=XPCSHELL_TEST_PROFILE_DIR&path=&case=true&regexp=false), and I can picture a few ways this would make the browser work in unexpected ways.

Hey Nicolas,

Due to Bug 1598562, we need to create the system variable in order for the "services.settings.server" to be read on Beta and Release.

That's interesting!
mh, so I'm not sure what the next steps should be.
We could look at the actual value of the env variable and allow it if it has a given value we agree on (something more unique than foo).
Or we could check that the value is something that looks like what is actually set from the xpcshell test harness (that might be tricky depending on platforms, remote, future changes, ...)
Or we could rely on something else to indicate that we're running an xpcshell test.

ccing Julian on this as he has experience about devtools and xpcshell tests.

Nicolas, what about also checking XPCSHELL_TEST_TEMP_DIR env variable?
It seems to be set as much as we set XPCSHELL_TEST_PROFILE_DIR:
https://searchfox.org/mozilla-central/source/testing/xpcshell/runxpcshelltests.py#728-731

        self.profileDir = self.setupProfileDir()
        self.tempDir = self.setupTempDir()

 def setupProfileDir(self):
        ...
        self.env["XPCSHELL_TEST_PROFILE_DIR"] = profileDir

def setupTempDir(self):
        tempDir = mkdtemp(prefix='xpc-other-', dir=self._rootTempDir)
        self.env["XPCSHELL_TEST_TEMP_DIR"] = tempDir

There is probably some other ways to detect xpcshell... but well... if we want to keep hacking env variable!

Otherwise, if we do support only xpcshell test and not any xpcshell run (which I think is the case),
we can hack around something over there:
https://searchfox.org/mozilla-central/source/testing/xpcshell/head.js#469-513
And do something to significate to DevTools code that we are in xpcshell.

Set release status flags based on info from the regressing bug 1642295

Set release status flags based on info from the regressing bug 1642295