https://treeherder.mozilla.org/#/jobs?filter-searchStr=autophone&exclusion_profile=false&repo=b2g-inbound&revision=4e5f3ed0c573 showed no measurements for remote nytimes s1s2 but looking at the logcat I do see Throbber start, stop.

ditto for https://treeherder.mozilla.org/#/jobs?filter-searchStr=autophone&exclusion_profile=false&repo=b2g-inbound&revision=4e5f3ed0c573 remote twitter s1s2. Note there is a problem in the logcat collection where some content is repeated: search for 09-17 13:22:19.210 in https://autophone.s3.amazonaws.com/pub/mozilla.org/mobile/tinderbox-builds/b2g-inbound-android-api-11/1442495394/autophone-s1s2-s1s2-twitter-remote.ini-1-nexus-5-kot49h-5-d5990586-1edd-43d8-9fdf-aef742f7c9fb-logcat.log That block is repeated 30 times. The same story is in play for 09-17 12:57:13.340 in https://autophone.s3.amazonaws.com/pub/mozilla.org/mobile/tinderbox-builds/b2g-inbound-android-api-11/1442495394/autophone-s1s2-s1s2-nytimes-remote.ini-1-nexus-5-kot49h-5-1037f9f6-7c1f-42c1-923d-78c98cbe3560-logcat.log This repetition existed prior to bug 1205388 since I can reproduce it from a run which occurred prior to the landing but it is much worse now. See https://treeherder.mozilla.org/#/jobs?repo=b2g-inbound&revision=33ab2e156c30&filter-searchStr=autophone&exclusion_profile=false and https://autophone.s3.amazonaws.com/pub/mozilla.org/mobile/tinderbox-builds/b2g-inbound-android-api-11/1442354975/autophone-s1s2-s1s2-twitter-remote.ini-1-nexus-4-jdq39-4-91520d55-80a8-48b1-ba72-f2798ca1f374-logcat.log But the failure to detect the measurements is not a regression since that occurred in the earlier run as well.

Another example on mozilla-beta https://treeherder.mozilla.org/#/jobs?filter-searchStr=autophone&exclusion_profile=false&repo=mozilla-beta&revision=4d92cee6e6b9 https://autophone.s3.amazonaws.com/pub/mozilla.org/mobile/tinderbox-builds/mozilla-beta-android-api-11/1443090737/autophone-s1s2-s1s2-nytimes-remote.ini-1-nexus-4-jdq39-4-826e704a-1991-4236-b547-f49a8b6b4611-autophone.log throbber start was detected but not throbber stop. Looking at the logcat https://autophone.s3.amazonaws.com/pub/mozilla.org/mobile/tinderbox-builds/mozilla-beta-android-api-11/1443090737/autophone-s1s2-s1s2-nytimes-remote.ini-1-nexus-4-jdq39-4-826e704a-1991-4236-b547-f49a8b6b4611-logcat.log I see examples of where Throbber stop is missing due to fennec being killed via a SIGTERM which is what snorp was referring to in bug 1177485. 09-24 09:53:42.892 I/GeckoToolbarDisplayLayout(21361): zerdatime 1257363 - Throbber start 09-24 09:53:42.892 I/ActivityManager( 557): Displayed org.mozilla.firefox_beta/.App: +866ms 09-24 09:53:42.982 I/GeckoHealth(21361): firefox_beta :: HealthReportBroadcastService :: Registering HealthReportUploadService. 09-24 09:53:42.982 I/GeckoHealth(21361): firefox_beta :: BackgroundService :: Setting inexact repeating alarm for interval 3600000 ... 09-24 09:54:44.037 I/Gecko (21361): 1443113684030 addons.manager WARN Exception calling callback: [Exception... "aUri must be a non-empty string" nsresu 09-24 09:56:25.135 D/Zygote ( 159): Process 21361 terminated by signal (15) Looking through the log, it appears the addons.manager WARN always preceeds the SIGTERM which kills fennec. The full message is: 09-24 09:53:42.892 I/GeckoToolbarDisplayLayout(21361): zerdatime 1257363 - Throbber start 09-24 09:54:44.037 I/Gecko (21361): 1443113684030 addons.manager WARN Exception calling callback: [Exception... "aUri must be a non-empty string" nsresult: "0x80070057 (NS_ERROR_ILLEGAL_VALUE)" location: "JS frame :: resource://gre/modules/AddonManager.jsm :: AMI_escapeAddonURI :: line 160" data: no] Stack trace: AMI_escapeAddonURI()@resource://gre/modules/AddonManager.jsm:160 < AM_escapeAddonURI()@resource://gre/modules/AddonManager.jsm:371 < escapeAddonURI()@resource://gre/modules/addons/XPIProvider.jsm:188 < UpdateChecker()@resource://gre/modules/addons/XPIProvider.jsm:844 < AddonWrapper_findUpdates()@resource://gre/modules/addons/XPIProvider.jsm:991 < aus_notify/</<()@jar:jar:file:///data/app/org.mozilla.firefox_beta-1.apk!/assets/omni.ja!/components/AddonUpdateService.js:9 < forEach()@self-hosted:211 < aus_notify/<()@jar:jar:file:///data/app/org.mozilla.firefox_beta-1.apk!/assets/omni.ja!/components/AddonUpdateService.js:9 < safeCall()@resource://gre/modules/AddonManager.jsm:18 < getAddonsByTypes_noMoreObjects()@resource://gre/modules/ 09-24 09:56:25.135 D/Zygote ( 159): Process 21361 terminated by signal (15) the SIG 9/SIGKILLs are from Quitter shutting down the browser which are expected. question 1: Where and why are the SIGTERMs issued? Is Autophone responsible? We are using an old version of Quitter which is not the version in the tree. I've been meaning to replace the version Autophone uses with the version in the tree, but haven't gotten to it yet. We need to do that in preparation for extension signing anyway. question 2: Why isn't Autophone detecting any measurements since there were at least a few. snorp: what is your opinion of the addons.manager causing the SIGTERM to be issued?

https://treeherder.mozilla.org/#/jobs?filter-searchStr=autophone&exclusion_profile=false&repo=mozilla-inbound&revision=793589a533f1 appears to be the same issue.

The SIGTERM issue is unrelated. I see this frequently and it is not related to missing the measurement.

This is a belt and braces patch. It shouldn't really have an effect since there should not ever be a case where the Throbber start or browser chrome startup times would be seen twice, but if they were this will discard the first and use the second. It also changes a couple of unable to find measurement log messages from info to warning.

Comment on attachment 8667408 [details] [diff] [review] bug-1205836-reset-throbber-chrome-start-times-v1.patch Review of attachment 8667408 [details] [diff] [review]: ----------------------------------------------------------------- great to see more debugging!

The "normal" way to suppress some external network accesses is to set a preference to disable a feature and also set the url associated with the external network access to a null string. This can cause problems if Firefox checks the value for validity. For example, setting the addon update url to '' results in 09-27 08:18:05.627 E/GeckoConsole( 2876): [JavaScript Error: "1443367085635 Toolkit.GMP ERROR GMPInstallManager.simpleCheckAndInstall Could not check for addons: [Exception... "The URI is malformed" nsresult: "0x804b000a (NS_ERROR_MALFORMED_URI)" location: "JS frame :: resource://gre/modules/addons/ProductAddonChecker.jsm :: downloadXML/< :: line 81" data: no] Stack trace: downloadXML/<()@resource://gre/modules/addons/ProductAddonChecker.jsm:81 < downloadXML()@resource://gre/modules/addons/ProductAddonChecker.jsm:74 < ProductAddonChecker.getProductAddonList()@resource://gre/modules/addons/ProductAddonChecker.jsm:299 < GMPInstallManager.prototype.checkForAddons()@resource://gre/modules/GMPInstallManager.jsm:107 < GMPInstallManager.prototype.simpleCheckAndInstall<()@resource://gre/modules/GMPInstallManager.jsm:213 < next()@self-hosted:670 < TaskImpl_run()@resource://gre/modules/Task.jsm:314 < TaskImpl()@resource://gre/modules/Task.jsm:275 < createAsyncFunction/asyncFunction()@resource://gre/modules/Task.jsm:249 < startup/BrowserApp_delayedStartup/< This patch uses urls pointing to localhost as well as using a data url for media.gmp-manager.url.override. It suppresses all of the internal JavaScript Errors I have seen with the exception of webappstartup's Toolkit.Telemetry ERROR ClientID::getCachedClientID - invalid client id in preferences, resetting. This is because we have to start the webapp at least once in order to create the profile we will use before we can set the preferences in the profile. It is hoped that eliminating unnecessary exceptions in Firefox will help reduce failures but we will only know more once this is in production.

Comment on attachment 8667415 [details] [diff] [review] bug-1205836-suppress-errors-due-to-invalid-pref-values-v1.patch Review of attachment 8667415 [details] [diff] [review]: ----------------------------------------------------------------- it would be nice to see this use the same prefs file as we do from in-tree; for now, this probably solves a lot of problems. ::: phonetest.py @@ +420,4 @@ > 'browser.trackingprotection.gethashURL': '', > 'browser.trackingprotection.updateURL': '', > + 'browser.translation.bing.authURL': 'http://localhost/browser/browser/components/translation/test/bing.sjs', > + 'browser.translation.bing.translateArrayURL': 'http://localhost/browser/browser/components/translation/test/bing.sjs', do we really have bing.sjs setup?

re bing, for us in Autophone land? no. None of these actually are loaded with the exception of the media gmp override which is why I set it to an empty <updates/> document. I've been running tests with this since Friday night I think and have not seen any hits in my local web server logs due to these. Regarding the in tree stuff, it appears we set these types of prefs in several files some of which I've listed in the code. It would be great if we had one location or even one pref so we didn't have to chase these down whenever someone adds a new external network access. Note Toolkit.GMP ERROR GMPInstallManager.simpleCheckAndInstall occurs in non-Autophone android tests: See http://mozilla-releng-blobs.s3.amazonaws.com/blobs/mozilla-inbound/sha512/9809d57edbabdf5322eb152a510c53ce186300d0b630cc6464a0273d6f9bfa7a3a1944e75dd07bed5a64a048f6b02883b47a26b638f3419a2426f72c9574bfa1

there are .sjs references in the prefs, I wanted to see if we had .sjs stuff running. If not, we should just make it a generic 404 page that we hit or some page that loads.

I used the original prefs which referenced urls and changed their %(server)s to localhost and left the url paths as is in order to be able to determine which url was the culprit if the url was actually dereferenced. The idea is that if any of these actually try to dereference the url, it will hit a 404 if there is a local web server running on port 80. If there isn't a local web server running on port 80 it will just fail immediately with a Unable to connect error. I thought this approach would work whether or not we are guaranteed to have local web server running.

I think it will work, it would be nice to make it work smoother though :)

Yeah, I'd like that too. The whole topic is whack-a-mole.

https://github.com/mozilla/autophone/commit/f158c591a926499d7ad129cc346a8212352bc196 https://github.com/mozilla/autophone/commit/b2bbbab74f507b77f74b6fbb3908e196b17036db Leaving open for further investigations.

Autophone is going away. Resolving these to wontfix.