Starting with the nightlies built on Tuesday, a lot of the mochitests that run against the Nightly builds have been failing after trying to contact tiles.services.mozilla.com: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&fromchange=56c6c81993e6b0f0e094e0c88b42ce3ec566ead9&noautoclassify&filter-searchStr=nightly%20(bc&group_state=expanded&selectedJob=115339579 I suspect something related to activity stream or the new tab page, since they're touching tiles-related stuff.

The add-on does pref things differently for local/"default"-channel builds vs official builds. I'm assuming there's a pref file somewhere for these "nightly" channel build test runs ?

[Tracking Requested - why for this release]: This is causing severely degraded test coverage on Nightly builds at the moment.

(In reply to Ed Lee :Mardak from comment #1) > The add-on does pref things differently for local/"default"-channel builds > vs official builds. I'm assuming there's a pref file somewhere for these > "nightly" channel build test runs ? That sounds exactly like what I've been suspecting. Would explain why only nightly builds are affected and not regular CI builds. We set prefs for our CI profiles in a few different places, but the most applicable one for this issue is at: https://dxr.mozilla.org/mozilla-central/source/testing/profiles/prefs_general.js

Is there a way to `try` as "nightly" builds? I suppose I could somewhere force the channel to be "nightly" to simulate. I believe we'll want to do ```diff diff --git a/testing/profiles/prefs_general.js b/testing/profiles/prefs_general.js --- a/testing/profiles/prefs_general.js +++ b/testing/profiles/prefs_general.js @@ -299,2 +299,3 @@ user_pref("browser.aboutHomeSnippets.updateUrl", "nonexistent://test"); user_pref("browser.newtabpage.activity-stream.default.sites", ""); +user_pref("browser.newtabpage.activity-stream.telemetry", false); ```

For Linux and OSX, they're scheduled as regular Taskcluster jobs and should be testable on Try with normal syntax - |try: -b do -p linux64-nightly -u mochitest-bc -t none|. Worst case, you should be able to add them to existing Try push via the Add New Jobs option.

Try kinda running here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=5010cc030ff751bd377c82cd78db94d50bfda4a3

You probably can't use artifact builds for this...

(In reply to Ryan VanderMeulen [:RyanVM] from comment #8) > You probably can't use artifact builds for this... Ha ha. Doh. |mach try| foiled no more! Bug 1382827 :) https://reviewboard.mozilla.org/r/159484/

Comment on attachment 8888438 [details] Bug 1382719 - Mochitests against Nightly builds are mostly failing with telemetry connections. Looks like we add the browser.newtabpage.activity-stream.default.sites pref change to testing/talos/talos/config.py as well. Joel, should we add this pref change there too?

A quick glance shows no talos running on Nightly builds. I would guess talos is running on "default" channel like a local build, so the `telemetry` pref is already false. We've been getting talos results with activity stream enabled. https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=nightly%20talos

thanks for the heads up- we should add any pref changes to mochitest to talos testing/talos/talos/config.py

Comment on attachment 8888438 [details] Bug 1382719 - Mochitests against Nightly builds are mostly failing with telemetry connections. Try push is green, my hero <3

Comment on attachment 8888438 [details] Bug 1382719 - Mochitests against Nightly builds are mostly failing with telemetry connections. https://reviewboard.mozilla.org/r/159388/#review164868 Looks good; r=dmose.

Pushed by ryanvm@gmail.com: https://hg.mozilla.org/mozilla-central/rev/7d2e89fb9233 Disable Activity Stream telemetry during test runs to avoid nonlocal connection crashes. r=dmose, a=RyanVM

Comment on attachment 8888438 [details] Bug 1382719 - Mochitests against Nightly builds are mostly failing with telemetry connections. https://reviewboard.mozilla.org/r/159388/#review164872 r=dmose

Comment on attachment 8888438 [details] Bug 1382719 - Mochitests against Nightly builds are mostly failing with telemetry connections. https://reviewboard.mozilla.org/r/159390/#review164876

Not even sure how it's possible, but looks like this is also happening on esr52: https://treeherder.mozilla.org/#/jobs?repo=mozilla-esr52&selectedJob=116138221

Sorry that's a different IP.. Guess it could somehow still be related to this.

browser/extensions/activity-stream doesn't even exist there: https://hg.mozilla.org/releases/mozilla-esr52/file/c52d8ce8d5dfd31e41dd4957428d85d5869e9a72/browser/extensions dmose and I have seen this as of today when sherrifing our `pine` repository, and we were trying to track that down to see if one of our recent activity stream commits caused it. We've only seen it once so far. Also on osx opt + debug: https://treeherder.mozilla.org/#/jobs?repo=pine&revision=58faa12409fe273e163b532e629742ef20602bce&filter-classifiedState=unclassified&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&selectedJob=116029827

ahal: see bug 1382858

Verified fixed on today's nightlies \m/ :-). Thanks again for jumping on this, Ed!