Since bug 706340 landed in mozilla-central, we're seeing this failure: TEST-UNEXPECTED-FAIL | /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetryPing.js | false == true - See following stack: JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/xpcshell/head.js :: do_throw :: line 462 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/xpcshell/head.js :: _do_check_eq :: line 556 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/xpcshell/head.js :: do_check_eq :: line 577 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/xpcshell/head.js :: do_check_true :: line 591 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetryPing.js :: checkHistograms :: line 85 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/Daily.app/Contents/MacOS/components/httpd.js :: <TOP_LEVEL> :: line 2272 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/Daily.app/Contents/MacOS/components/httpd.js :: <TOP_LEVEL> :: line 1169 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/Daily.app/Contents/MacOS/components/httpd.js :: <TOP_LEVEL> :: line 1617 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/Daily.app/Contents/MacOS/components/httpd.js :: <TOP_LEVEL> :: line 1465 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/Daily.app/Contents/MacOS/components/httpd.js :: <TOP_LEVEL> :: line 1334 TEST-INFO | (xpcshell/head.js) | exiting test TEST-INFO | (xpcshell/head.js) | test 1 finished TEST-INFO | (xpcshell/head.js) | exiting test TEST-UNEXPECTED-FAIL | /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetryPing.js | false == true - See following stack: JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/xpcshell/head.js :: do_throw :: line 462 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/xpcshell/head.js :: _do_check_eq :: line 556 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/xpcshell/head.js :: do_check_eq :: line 577 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/xpcshell/head.js :: do_check_true :: line 591 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/xpcshell/tests/toolkit/components/telemetry/tests/unit/test_TelemetryPing.js :: <TOP_LEVEL> :: line 195 JS frame :: /builds/slave/comm-central-macosx64-opt-unittest-xpcshell/build/xpcshell/head.js :: _execute_test :: line 363 JS frame :: -e :: <TOP_LEVEL> :: line 1 It looks like: Cc["@mozilla.org/gfx/info;1"].getService(Ci.nsIGfxInfo).adapterVendorID (and .adapterDeviceID) are both null for us. I don't understand why that would be. Could we be having an issue if graphics acceleration is turned off by default?

Ok, I get this if I sudo to another user on my mac and run the tests - hence the sudo shell isn't able to access the display drivers. John/Gozer, can you guys have a look and see if we're running our Mac setup differently to Firefox somehow? (this is only seen on Mac). I do find it a bit strange that MozMill tests are fine, but xpcshell isn't though.

(In reply to Mark Banner (:standard8) from comment #1) > Ok, I get this if I sudo to another user on my mac and run the tests - hence > the sudo shell isn't able to access the display drivers. > > John/Gozer, can you guys have a look and see if we're running our Mac setup > differently to Firefox somehow? (this is only seen on Mac). > > I do find it a bit strange that MozMill tests are fine, but xpcshell isn't > though. Any idea if it's possible to detect that situation? We can just pull the check for those two values but it would be nice to have it tested somewhere (it's already limited to osx/win for the same reason as this bug).

(In reply to John Daggett (:jtd) from comment #2) > Any idea if it's possible to detect that situation? We can just pull the > check for those two values but it would be nice to have it tested somewhere > (it's already limited to osx/win for the same reason as this bug). I don't know of any way. I'm not sure we're really designed to run without a window ;-)

I'm currently trying to reproduce this on staging and will see if I can find a fix.

Thunderbird disables hardware acceleration, and I'm pretty sure that results in us never initializing GfxInfo, so we never add these fields to Telemetry. The best way to fix this is to always initialize GfxInfo, I think. Is there any reason for us not to do that?

I pushed this to Thunderbird try: pref("layers.acceleration.disabled", true); pref("layers.acceleration.disabled", false); but it didn't fix it, does it need something else as well? I also could reproduce this failure on my Mac locally. - If I ran the tests in my normal development login, then they passed. - However if I logged into a different account, used su to get to my development account, and then ran them, they failed. Hence why I was suspecting more of a builder issue than a code issue.

Latest debug from try server: Init GfxInfoBase GfxInfo::Init Getting device info Vendor: , Device: So for some reason we're not getting the device info when we should. gozer tells me that the builders are from the exact same system image that Firefox uses, so there shouldn't be an issue there. My only thought currently, is that as we're testing the telementry ping, and not the graphics info, is how would people feel about inserting a fake gfxInfo for debug mode tests, the same way as is done for some other tests? For example: http://mxr.mozilla.org/comm-central/source/mozilla/toolkit/mozapps/extensions/test/xpcshell/test_gfxBlacklist_Equal_OK.js#36

I just noticed SeaMonkey is seeing this as well. So this definitely isn't hardware accel, as they have it enabled by default. Looking back at out 10.5 builders now that we have had lots more builds over the week - tb2-darwin9-slave70 is passing, the rest aren't. Hence this does look more like a buildbot config issue. I wonder if there's a setting or something in Mac that could cause this? I'll talk with gozer and jhopkins about this later.

Just decided to compare tb2-darwin9-slave70 with tb2-darwin9-slave72, and after removing the cruft, there are some differences, looks like. Library/ColorSync/Profiles/Displays/Display-5B81C5C0.icc Library/Keychains/System.keychain Library/LaunchDaemons/com.reductivelabs.puppet.plist Library/Logs/DirectoryService/DirectoryService.server.log Library/Logs/DirectoryService/DirectoryService.server.log.1 Library/Logs/DirectoryService/DirectoryService.server.log.2 Library/Logs/DirectoryService/DirectoryService.server.log.3 Library/Logs/DirectoryService/DirectoryService.server.log.4 Library/Logs/DirectoryService/DirectoryService.server.log.5 Library/Logs/DirectoryService/DirectoryService.server.log.6 Library/Logs/DirectoryService/DirectoryService.server.log.7 Library/Logs/DirectoryService/DirectoryService.server.log.8 Library/Preferences/.GlobalPreferences.plist Library/Preferences/DirectoryService/.DSIsRunning Library/Preferences/SystemConfiguration/NetworkInterfaces.plist Library/Preferences/SystemConfiguration/com.apple.network.identification.plist Library/Preferences/SystemConfiguration/com.apple.smb.server.plist Library/Preferences/SystemConfiguration/preferences.plist Library/Preferences/SystemConfiguration/preferences.plist.old Library/Preferences/cfx#61mDDsC Library/Preferences/cfx#69gaWVw Library/Preferences/cfx#75noL37 Library/Preferences/cfx#FxSOI14 Library/Preferences/cfx#M1oGY2E Library/Preferences/cfx#TTK2qj5 Library/Preferences/cfx#VeXMjeV Library/Preferences/cfx#hCBeRF9 Library/Preferences/cfx#zqUkSIZ Library/Preferences/com.apple.ARDAgent.plist Library/Preferences/com.apple.AppleFileServer.plist Library/Preferences/com.apple.BezelServices.plist Library/Preferences/com.apple.Bluetooth.plist Library/Preferences/com.apple.RemoteManagement.launchd Library/Preferences/com.apple.RemoteManagement.plist Library/Preferences/com.apple.TimeMachine.plist Library/Preferences/com.apple.VNCSettings.txt Library/Preferences/com.apple.audio.DeviceSettings.plist Library/Preferences/com.apple.audio.SystemSettings.plist Library/Preferences/com.apple.loginwindow.plist Library/Preferences/com.apple.security.systemidentities.plist Library/Receipts/clang-3.0-r135723.moz0.pkg/Contents/Info.plist Library/Receipts/repackaging-tools-10.5.pkg/Contents/Info.plist Library/Receipts/simplejson-2.1.1-py25.pkg/Contents/Info.plist Library/Receipts/yasm-1.1.0.pkg/Contents/Info.plist System/Library/CoreServices/.disk_label System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/UserDict.pyc System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/codecs.pyc System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/copy_reg.pyc System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/encodings/__init__.pyc System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/encodings/aliases.pyc System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/encodings/ascii.pyc System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/linecache.pyc System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/os.pyc System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/posixpath.pyc System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site.pyc System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/stat.pyc System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/types.pyc System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/warnings.pyc System/Library/LaunchDaemons/com.apple.WindowServer.plist System/Library/LaunchDaemons/com.apple.blued.plist System/Library/LaunchDaemons/com.apple.kdcmond.plist System/Library/LaunchDaemons/com.apple.loginwindow.plist Users/cltbld/.bash_history Users/cltbld/.lesshst Users/cltbld/.ssh/known_hosts Users/cltbld/.viminfo Users/cltbld/Library/Application Support/AddressBook/.database.lockN Users/cltbld/Library/Application Support/AddressBook/Metadata/.info Users/cltbld/Library/Logs/DiscRecording.log Users/cltbld/Library/Logs/fsck_hfs.log Users/cltbld/Library/Preferences/ByHost/.GlobalPreferences.0016cbae26ff.plist Users/cltbld/Library/Preferences/ByHost/com.apple.Bluetooth.0016cbae26ff.plist Users/cltbld/Library/Preferences/ByHost/com.apple.SoftwareUpdate.0016cbae26ff.plist Users/cltbld/Library/Preferences/ByHost/com.apple.iCal.helper.0016cbae26ff.plist Users/cltbld/Library/Preferences/ByHost/com.apple.preference.displays.0016cbae26ff.plist Users/cltbld/Library/Preferences/ByHost/com.apple.scheduler.0016cbae26ff.plist Users/cltbld/Library/Preferences/ByHost/com.apple.screensaver.0016cbae26ff.plist Users/cltbld/Library/Preferences/ByHost/com.apple.systempreferences.0016cbae26ff.plist Users/cltbld/Library/Preferences/ByHost/com.apple.windowserver.0016cbae26ff.plist Users/cltbld/Library/Preferences/ByHost/pbs.0016cbae26ff.plist Users/cltbld/Library/Preferences/QuickTime Preferences Users/cltbld/Library/Preferences/com.apple.BezelServices.plist Users/cltbld/Library/Preferences/com.apple.DownloadAssessment.plist Users/cltbld/Library/Preferences/com.apple.LaunchServices.plist Users/cltbld/Library/Preferences/com.apple.PreferenceSync.plist Users/cltbld/Library/Preferences/com.apple.PubSubAgent.plist Users/cltbld/Library/Preferences/com.apple.desktop.plist Users/cltbld/Library/Preferences/com.apple.dock.db Users/cltbld/Library/Preferences/com.apple.dock.plist Users/cltbld/Library/Preferences/com.apple.finder.plist Users/cltbld/Library/Preferences/com.apple.internetconfigpriv.plist Users/cltbld/Library/Preferences/com.apple.quicktime.plugin.preferences.plist Users/cltbld/Library/Preferences/com.apple.recentitems.plist Users/cltbld/Library/Preferences/com.apple.spotlight.plist Users/cltbld/Library/Preferences/com.apple.systempreferences.plist Users/cltbld/Library/Preferences/com.apple.systemuiserver.plist Users/cltbld/Library/Preferences/org.mozilla.crashreporter.plist Users/cltbld/Library/Preferences/org.mozilla.daily.plist Users/cltbld/Library/Preferences/org.mozilla.dailydebug.plist Users/cltbld/Library/Preferences/org.mozilla.earlybird.plist Users/cltbld/Library/Preferences/org.mozilla.earlybirddebug.plist Users/cltbld/Library/Preferences/org.mozilla.thunderbird.plist Users/cltbld/Library/PubSub/Database/Database.sqlite3 Users/cltbld/Library/Thunderbird/console.log Users/cltbld/Library/Thunderbird/profiles.ini Users/cltbld/hg opt/local/etc/cacert.pem opt/local/etc/wgetrc usr/X11/man/whatis usr/local/bin/addon_id usr/local/bin/check-for-package.sh usr/local/bin/jsbridge usr/local/bin/mozmill usr/local/bin/mozmill-restart usr/local/bin/mozmill-thunderbird usr/local/bin/mozrunner usr/local/bin/runslave.py usr/local/bin/setup-configuration.sh usr/local/bin/setup-nagios-user.sh usr/local/bin/sleep-and-run-puppet.sh usr/local/share/man/whatis usr/share/man/whatis

Turning off vnc on the builders hasn't worked. I'm currently pondering if there's other bits we can check.

(In reply to Mark Banner (:standard8) from comment #8) > I just noticed SeaMonkey is seeing this as well. So this definitely isn't > hardware accel, as they have it enabled by default. > SeaMonkey failures due to hwaccel is not uncommon (our plan for these hwaccel issues is to disable the test/fix the test), Our mac slaves are currently a mix of r3/r2 mini's and all setup as builders. I don't think we even have any of the hwaccel graphic cards installed, but I can't recall for certain.

I'm currently testing this on try servers. The basic idea for the patch is that with the telemetry ping test we are not trying to test obtaining the information from the hardware. Therefore we use the fake gfx info available in debug mode and if we're in release, then we don't test.

Try run for ce232a5245c6 is complete. Detailed breakdown of the results available here: https://tbpl.mozilla.org/?tree=Try&rev=ce232a5245c6 Results (out of 26 total builds): exception: 1 success: 23 warnings: 2 Builds (or logs if builds failed) available at: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/bugzilla@standard8.plus.com-ce232a5245c6

Comment on attachment 596151 [details] [diff] [review] Possible fix John/Joe, how do you feel about this patch? I think it just does enough to make sure we get the result out, whilst not depending on the hardware available for the test.

Landed on inbound: https://hg.mozilla.org/integration/mozilla-inbound/rev/37cecf145092

Comment on attachment 596151 [details] [diff] [review] Possible fix [Approval Request Comment] Regression caused by (bug #): bug 706340 User impact if declined: None Testing completed (on m-c, etc.): About 2 days Risk to taking this patch (and alternatives if risky): Test-only change, no risk String changes made by this patch: none This is a test-only change to fix some permanent orange for c-c apps caused by assuming that we have hardware accelerated cards on all machines, when we don't need to be testing that here.

Comment on attachment 596151 [details] [diff] [review] Possible fix [Triage Comment] Test-only fix, approved for Aurora 12.

Landed in aurora: https://hg.mozilla.org/releases/mozilla-aurora/rev/dfb7c3567c49