Filed by: apavel [at] mozilla.com https://treeherder.mozilla.org/logviewer.html#?job_id=171710486&repo=mozilla-inbound https://queue.taskcluster.net/v1/task/S4u57933REiFkOMexVSeMw/runs/0/artifacts/public/logs/live_backing.log 13:17:29 INFO - PID 1714 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath 13:17:30 INFO - PID 1714 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath 13:17:30 INFO - PID 1714 | 2018-04-03 13:17:30.911 plugin-container[1719:7793] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x9543, name = 'com.apple.tsm.portname' 13:17:30 INFO - PID 1714 | See /usr/include/servers/bootstrap_defs.h for the error codes. 13:17:31 INFO - PID 1714 | 1522786651177 Marionette INFO Listening on port 2828 13:17:31 INFO - TEST-START | /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html 13:17:31 INFO - PID 1714 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath 13:17:31 INFO - PID 1714 | 1522786651932 Marionette INFO Testing http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html == http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/reference/shape-outside-linear-gradient-001-ref.html 13:17:31 INFO - PID 1714 | 2018-04-03 13:17:31.973 plugin-container[1720:7886] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x9543, name = 'com.apple.tsm.portname' 13:17:31 INFO - PID 1714 | See /usr/include/servers/bootstrap_defs.h for the error codes. 13:17:32 INFO - PID 1714 | 1522786652094 Marionette INFO Found 20859 pixels different, maximum difference per channel 255 13:17:32 INFO - TEST-UNEXPECTED-FAIL | /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html | Testing http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html == http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/reference/shape-outside-linear-gradient-001-ref.html 13:17:32 INFO - TEST-INFO took 647ms 13:17:32 INFO - PID 1714 | *** UTM:SVC TimerManager:registerTimer called after profile-before-change notification. Ignoring timer registration for id: telemetry_modules_ping 13:17:32 INFO - PID 1714 | 13:17:32 INFO - PID 1714 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost 13:17:32 INFO - PID 1714 | 13:17:32 WARNING - u'runner_teardown': () 13:17:32 INFO - certutil | 13:17:32 INFO - certutil | 13:17:32 INFO - certutil | 13:17:32 INFO - Certificate Nickname Trust Attributes 13:17:32 INFO - SSL,S/MIME,JAR/XPI 13:17:32 INFO - 13:17:32 INFO - web-platform-tests CT,, 13:17:32 INFO - 13:17:33 INFO - PID 1724 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath 13:17:33 INFO - PID 1724 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath 13:17:34 INFO - PID 1724 | 2018-04-03 13:17:34.280 plugin-container[1726:7997] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x973b, name = 'com.apple.tsm.portname' 13:17:34 INFO - PID 1724 | See /usr/include/servers/bootstrap_defs.h for the error codes. 13:17:34 INFO - PID 1724 | 1522786654586 Marionette INFO Listening on port 2828 13:17:34 INFO - TEST-START | /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html 13:17:35 INFO - PID 1724 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath 13:17:35 INFO - PID 1724 | 1522786655287 Marionette INFO Testing http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html == http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/reference/shape-outside-linear-gradient-001-ref.html 13:17:35 INFO - PID 1724 | 2018-04-03 13:17:35.327 plugin-container[1727:8078] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x974b, name = 'com.apple.tsm.portname' 13:17:35 INFO - PID 1724 | See /usr/include/servers/bootstrap_defs.h for the error codes. 13:17:35 INFO - PID 1724 | 1522786655440 Marionette INFO Found 20859 pixels different, maximum difference per channel 255 13:17:35 INFO - TEST-UNEXPECTED-FAIL | /css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html | Testing http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/shape-outside-linear-gradient-005.html == http://web-platform.test:8000/css/css-shapes/shape-outside/shape-image/gradients/reference/shape-outside-linear-gradient-001-ref.html

Looks like this is a real failure, caused by bug 1404222. (The final cset there removed some failure annotations for some of these tests, but it looks like it still fails on this platform at least.)

(ni=bradwerth to figure out how best to sort this out)

Ahem is not loading for the new wpt reftests added in Bug 1404222. Both the test page and the reference page rely upon Ahem, and the failure log https://treeherder.mozilla.org/logviewer.html#?job_id=171710486&repo=mozilla-inbound&lineNumber=815 shows that the font is not rendering in either image. There's evidence that this has appeared before on this platform. https://searchfox.org/mozilla-central/source/testing/web-platform/meta/infrastructure/assumptions/ahem.html.ini shows that a recently-added wpt reftest that uses Ahem had to be marked expected fail on this platform. It's unclear why this is affecting new wpt reftests and not existing tests. Since Ahem.tff is available in the wpt subtree, my proposed approach is to add an explicit font-face declaration to the new tests; something like: @font-face { font-family: "Ahem"; src: url(/fonts/Ahem.ttf);

jgraham, see comment 3 -- do you know what might be going on here RE Ahem on this platform? (See the ahem.html.ini file that Brad linked, which seems to have been added in the "Update wpt metadata" commit in bug 1410245.)

[needinfo for comment 4]

Or it may be as simple as adding this declaration to the tests, since they appear in older, working tests: <meta name="flags" content="ahem"/>

So, Ahem on OSX is a problem. In theory wpt tests have system Ahem (I think this is not true for Mozilla reftests). And wptrunner has a --install-fonts option that's applied in prod. On OSX this copies Ahem.ttf to ~/Library/Fonts/ when the runner starts. On a local machine, this seems to work fine, but on our CI system Ahem still doesn't load. I have no idea what's going on, even though I got a loaner to investigate. A solution is to add Ahem directly to the puppet configs, but when I looked at that it was hard to figure out where it would go.

Maybe we should just skip these tests on OSX for the time being (via annotations in .ini files), then? I'm reluctant to add paper-over hacks to the test files themselves (particularly in the files written by other vendors, e.g. shape-outside-linear-gradient-001-ref.html), as the current Part 1 is kind of doing...

(jgraham, would you mind filing a bug to capture comment 11, if there isn't already one? Then we can reference that bug from .ini file annotations, perhaps, for tests like these & future ones that depend strongly on Ahem.)

(In reply to Daniel Holbert [:dholbert] (away 4/24 - 5/11) from comment #13) > (jgraham, would you mind filing a bug to capture comment 11, if there isn't > already one? Then we can reference that bug from .ini file annotations, > perhaps, for tests like these & future ones that depend strongly on Ahem.) Ah, I prematurely requested review for my fix, when the .ini files in there still contain a comment referencing THIS bug, which doesn't make any sense. I'll update those comments before landing, once we have a proper bug to reference.

Comment on attachment 8964721 [details] Bug 1451123: Disable some shape-outside image web-platform tests on macOS, since they fail without Ahem font support. https://reviewboard.mozilla.org/r/233446/#review239498

(In reply to Brad Werth [:bradwerth] from comment #16) > I'll update those comments before landing, once we have a proper bug to > reference. No need; this bug is fine to reference for now, I think. Better to fix the orange sooner, IMO; and curious people who stumble on the .ini file will still have a route to discovering whatever bug we end up spinning off of this one. (by reading comments here)

Pushed by bwerth@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/4b833df6ec89 Disable some shape-outside image web-platform tests on macOS, since they fail without Ahem font support. r=dholbert

Looks like all is well until we solve the root cause noted in comment 11.

Bug 1425698 is the Ahem-on-OSX issue. It's not 100% clear to me why you disabled the tests rather than marking them expected fail. The former makes it much less likely they we will rememeber to re-enable them.

Yes, please, let's mark it as failing instead?

(In reply to James Graham [:jgraham] from comment #22) > Bug 1425698 is the Ahem-on-OSX issue. Thanks! > It's not 100% clear to me why you disabled the tests rather than marking > them expected fail. That was my suggestion (comment 12) -- I was thinking that these were only failing on some OSX versions, since I only saw one row of test failures in the link from comment 0. If this is a known (and 100% reliable) issue on all OSX versions that we test in CI [and it sounds like it is?], then agreed, we should do expected-fail instead.

(I'd thought we still ran tests on multiple versions of OSX, but I guess we don't anymore -- only 10.10 now -- based on paging through treeherder backlog.)

(In reply to Emilio Cobos Álvarez [:emilio] from comment #23) > Yes, please, let's mark it as failing instead? Bug 1452184 has been opened to address this.