I am in the process of preparing tests to run on osx 10.14 in CI instead of 10.10. I found 4 tests that need attention.

here is a try job for test_tooltip.xul (a timeout):
https://treeherder.mozilla.org/#/jobs?repo=try&searchStr=mochitest-chrome&revision=b3b3319a7426c0e96b3d2c4c0d8106e95f920d23

here is a try job for test_findbar.xul and test_panel.xul:
https://treeherder.mozilla.org/#/jobs?repo=try&searchStr=mochitest-chrome&revision=c7df55609292b1af69338a9a1a3c49b346ce819a

here is a try push for the test_titlebar.xul:
https://treeherder.mozilla.org/#/jobs?repo=try&searchStr=mochitest-chrome&revision=7997b5ce9e54a0549821a63a131c935c14ccaef2&selectedJob=242696240

I am not sure if there is something in the OS settings/desktop, would be worth looking at.

Neil, could you take a look at these and help figure out if there is a fix or if we should disable the tests for osx 10.14.

It looks like two of the tests (test_panel.xul and test_titlebar.xul) fail waiting for the window to be moved. test_tooltip.xul also has a part of the test that moves the window, but I'm not clear from the log where the test is failing.

The first two tests have a block of code that disables that section on Linux, so it may be the case the test should be more robust or a section added for Mac.

I don't know anything about the test_findbar.xul test.

I found that test_findbar.xul is a perma fail and the others are intermittent- when running the tests solo, they are fine, so there is some interaction with other tests in the same manifest causing problems.

The priority flag is not set for this bug.
:enndeakin, could you have a look please?

For more information, please visit auto_nag documentation.

:enndeakin - it appears that these test failures are the only failing tests left in mochitest-chrome-2 for macosx1014; given that there exists some timeline to move to macosx1014, would it make sense to disable these tests for the time being?

I can create a patch that disables the parts of the test that fail as Linux does rather than disabling the test outright.

(In reply to Neil Deakin from comment #8)

I can create a patch that disables the parts of the test that fail as Linux does rather than disabling the test outright.

That sounds like a good plan. I'll wait for your patch to land, then re-test mochitest-chrome on macosx1014.

What magic incantation do I need to use to test 10.14 on try?

right now the tests are not able to run unless we edit the taskcluster definitions:
https://searchfox.org/mozilla-central/source/taskcluster/ci/test/test-sets.yml#325

you can see that mochitest-chrome is commented out, just remove the comment and then push to try :)

I use:
./mach try fuzzy -q 'test-macosx1014 mochitest-chrome'

:enndeakin - I'm following up regarding these tests for macosx1014. I'm trying to wrap up as many as I can this and next week - do you reckon a patch would be possible by then? If you don't have enough bandwidth then I am hoping to temporarily disable these tests temporarily for macosx1014 and migrate mochitest-chrome over.

I was only able to get to a point where test_panel.xul seemed not to have any more issues.

test_titlebar.xul seems to behave differently on 10.14 than 10.10 suggesting that there may be some actual different OS behaviour.

I haven't been able to fix the test_tooltip.xul test.

I'd suggest asking Mike de Boer about test_findbar.xul

I'll attach the patch I have so far, but I'd suggest that disabling on 10.14 would be ok for now.

:enndeakin - thanks for the partial patch, I will have the tests disabled for macosx1014 but with very explicit comments stating that this should not be the long-term.

I am not certain if related, but the URLbar is also seeing persistent failures on macosx1014 as well - bug 1558328.

Edit: also note that test_tooltiop.xul is currently passing on macosx1014.