https://tbpl.mozilla.org/php/getParsedLog.php?id=18219192&tree=Firefox NOISE: __FAILException caught: java.lang.NullPointerException__FAIL FIRE PROC: ' "MOZ_CRASHREPORTER_SHUTDOWN=1,MOZ_CRASHREPORTER_NO_REPORT=1,NO_EM_RESTART=1" am instrument -w -e deviceroot /mnt/sdcard/tests -e class org.mozilla.fennec.tests.testCheck org.mozilla.roboexample.test/org.mozilla.fennec.FennecInstrumentationTestRunner' NOISE: __FAILchecking that painted surface loaded: painted surface loaded__FAIL NOISE: Exception caught during test! NOISE: java.lang.NullPointerException NOISE: at org.mozilla.fennec.tests.PixelTest.loadAndPaint(PixelTest.java:23) NOISE: at org.mozilla.fennec.tests.testCheck.testCheck(testCheck.java:25) NOISE: at java.lang.reflect.Method.invokeNative(Native Method) NOISE: at java.lang.reflect.Method.invoke(Method.java:521) NOISE: at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204) NOISE: at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194) NOISE: at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186) NOISE: at org.mozilla.fennec.tests.BaseTest.runTest(BaseTest.java:122) NOISE: at junit.framework.TestCase.runBare(TestCase.java:127) NOISE: at junit.framework.TestResult$1.protect(TestResult.java:106) NOISE: at junit.framework.TestResult.runProtected(TestResult.java:124) NOISE: at junit.framework.TestResult.run(TestResult.java:109) NOISE: at junit.framework.TestCase.run(TestCase.java:118) NOISE: at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169) NOISE: at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154) NOISE: at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:520) NOISE: at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447) NOISE: __FAILException caught: java.lang.NullPointerException__FAIL NOISE: __startBeforeLaunchTimestamp1356264194664__endBeforeLaunchTimestamp NOISE: __startAfterTerminationTimestamp1356264229260__endAfterTerminationTimestamp Traceback (most recent call last): File "run_tests.py", line 261, in run_tests getting files in '/mnt/sdcard/tests/profile/minidumps/' Failed tcheckerboard: Stopped Sun, 23 Dec 2012 04:03:50 talos_results.add(mytest.runTest(browser_config, test)) File "/builds/tegra-087/talos-data/talos/ttest.py", line 387, in runTest test_results.add(browser_log_filename, counter_results=counter_results) File "/builds/tegra-087/talos-data/talos/results.py", line 120, in add browserLog = BrowserLogResults(filename=results, counter_results=counter_results, global_counters=self.global_counters) File "/builds/tegra-087/talos-data/talos/results.py", line 316, in __init__ self.error(match.group(1)) File "/builds/tegra-087/talos-data/talos/results.py", line 331, in error raise utils.talosError(message) talosError: 'checking that painted surface loaded: painted surface loaded [browser_output.txt]'

https://tbpl.mozilla.org/php/getParsedLog.php?id=18219649&tree=Firefox https://tbpl.mozilla.org/php/getParsedLog.php?id=18219236&tree=Firefox https://tbpl.mozilla.org/php/getParsedLog.php?id=18219186&tree=Firefox

This seemed to only happen on m-c for a brief period and did not occur on inbound...some sort of temporary merge problem?

We need to add this preference to talos: toolkit.telemetry.notifiedOptOut: 2

this really sucks that we cannot test this on try server. Is there a way to build 'nightly' builds on try server?

Doh, indeed, didn’t catch that on try... I thought every tests suite code were on m-c ? Also, if we don’t want any trouble in the future, we could do: + 'toolkit.telemetry.prompted': 999, + 'toolkit.telemetry.notifiedOptOut': 999, They did that for Thunderbird. How can I be sure that there is no other place where this pref is used in the tests?

999 is a good idea. There are a series of other test harnesses that we have, but not all run on our normal try server or are seen on tbpl. The biggest difference here was the regular vs nightly build.

(In reply to Joel Maher (:jmaher) from comment #28) > 999 is a good idea. There are a series of other test harnesses that we > have How can we found them? Until now, I only checked http://mxr.mozilla.org/mozilla-central/search?string=toolkit.telemetry.prompted , so this is why I missed Talos

for the more robust sustainable win!

hmm, using the values of 999 seem to cause my talos android tests to fail. switching back to a value of 2 :) http://hg.mozilla.org/build/talos/rev/d5cf61c4a4ec

(In reply to Joel Maher (:jmaher) from comment #35) > hmm, using the values of 999 seem to cause my talos android tests to fail. > switching back to a value of 2 :) Oh, I know why. It’s because of: http://mxr.mozilla.org/mozilla-central/source/browser/components/nsBrowserGlue.js#919 If we want to be able to fix tests in the long term, we should use if (telemetryDisplayed >== TELEMETRY_DISPLAY_REV) return; And same in browser.js for mobile. I will open a bug for that.

Joel, it’s weird, this is still happening, your patch should have already fixed it. Slaves needs to be restarted?

(In reply to Théo Chevalier [:tchevalier] from comment #49) > Joel, it’s weird, this is still happening, your patch should have already > fixed it. > Slaves needs to be restarted? The talos snapshot used by trunk needs to be updated (unlike releng bugs, talos bugs tend to get marked as fixed prior to deploy): https://hg.mozilla.org/mozilla-central/file/tip/testing/talos/talos.json

live on inbound, will be on m-c on the next merge