I am working on getting tests running in osx 10.14 and there are a handful of failures.

In this case browser_1119088.js is failing on both opt (bc3)/debug (bc4):
https://treeherder.mozilla.org/#/jobs?repo=try&searchStr=osx%2Cbrowser&revision=27d8e706de22ac93732dab0733347657e2a6433f&selectedJob=243484105

05:22:51 INFO - TEST-START | browser/components/shell/test/browser_1119088.js
05:22:52 INFO - Not taking screenshot here: see the one that was previously logged
05:22:52 INFO - Buffered messages logged at 05:22:51
05:22:52 INFO - Entering test bound
05:22:52 INFO - Buffered messages finished
05:22:52 INFO - TEST-UNEXPECTED-FAIL | browser/components/shell/test/browser_1119088.js | Desktop background database must exist for test to run. -
05:22:52 INFO - Stack trace:
05:22:52 INFO - chrome://mochikit/content/browser-test.js:test_ok:1314
05:22:52 INFO - chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js:null:23
05:22:52 INFO - resource://testing-common/BrowserTestUtils.jsm:withNewTab:111
05:22:52 INFO - chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js:null:9
05:22:52 INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1116
05:22:52 INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1144
05:22:52 INFO - chrome://mochikit/content/browser-test.js:nextTest/<:1005
05:22:52 INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
05:22:52 INFO - Not taking screenshot here: see the one that was previously logged
05:22:52 INFO - TEST-UNEXPECTED-FAIL | browser/components/shell/test/browser_1119088.js | Uncaught exception - [Exception... "Component returned failure code: 0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST) [nsIFile.copyTo]" nsresult: "0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST)" location: "JS frame :: chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js :: <TOP_LEVEL> :: line 30" data: no]
05:22:52 INFO - Stack trace:
05:22:52 INFO - @chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js:30:25
05:22:52 INFO - withNewTab@resource://testing-common/BrowserTestUtils.jsm:111:24
05:22:52 INFO - async*@chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js:9:26
05:22:52 INFO - Tester_execTest/<@chrome://mochikit/content/browser-test.js:1116:34
05:22:52 INFO - Tester_execTest@chrome://mochikit/content/browser-test.js:1144:12
05:22:52 INFO - nextTest/<@chrome://mochikit/content/browser-test.js:1005:14
05:22:52 INFO - SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:803:59
05:22:52 INFO - Leaving test bound
05:22:52 INFO - GECKO(1815) | MEMORY STAT vsizeMaxContiguous not supported in this build configuration.
05:22:52 INFO - GECKO(1815) | MEMORY STAT | vsize 6971MB | residentFast 351MB | heapAllocated 156MB
05:22:52 INFO - TEST-OK | browser/components/shell/test/browser_1119088.js | took 437ms
05:22:52 INFO - Not taking screenshot here: see the one that was previously logged
05:22:52 INFO - TEST-UNEXPECTED-FAIL | browser/components/shell/test/browser_1119088.js | Found an unexpected tab at the end of test run: about:logo -
05:22:52 INFO - checking window state

based on the failure message we fail a copyto here:
https://searchfox.org/mozilla-central/source/browser/components/shell/test/browser_1119088.js#30

desktopBackgroundDb.copyTo(null, desktopBackgroundDbBackup.leafName);

I assume we need to fix dirnames, permissions, or something else.

:mconley, I see you reviewed one of the patches in the original bug, any chance you could help out here or redirect me to someone who can?

Redirecting to spohl.

The test runs fine on my Mac 10.14 with an artifact build.

(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #0)

based on the failure message we fail a copyto here:
https://searchfox.org/mozilla-central/source/browser/components/shell/test/browser_1119088.js#30

The reason is just before that:

``
TEST-UNEXPECTED-FAIL | browser/components/shell/test/browser_1119088.js | Desktop background database must exist for test to run. -

This is expecting `~/Library/Application\ Support/Dock/desktoppicture.db` to exist and it isn't finding it.

I guess the 10.10 images/setup had the background changed at sometime, but maybe the newer 10.14 ones haven't? (and hence no database). Or there might be something else which triggeres the database creation (e.g. just entering the desktop settings maybe)?

interesting, lets see if we can get one set-

:dividehex- is it possible to set a custom/different background on the Mojave machines?

(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #4)

interesting, lets see if we can get one set-

:dividehex- is it possible to set a custom/different background on the Mojave machines?

Yes. Currently, the background is being set with a mobileconfig profile and the ability for the user to change it has been disabled. The background color/image that is used in 10.10 is not available on 10.14, so I picked a color that was available by default. Does the test expect a certain background color/img or is the test attempting to change it? We can either set it to what the test is expecting or allow the background to be changed by the user/test through a change in the mobileconfig profile.

Let me know what is needed.

:dividehex, does ~/Library/Application\ Support/Dock/desktoppicture.db exist? maybe the permissions are such that we cannot access it?

As it might help, here's what the test is doing:

• Checks there is an existing ~/Library/Application\ Support/Dock/desktoppicture.db
• Creates a backup copy of it (in the same directory called desktoppicture.db.backup)
• Sets the background image
• Checks that the background image has been saved to ~/Pictures
• Restores the backup database
• Restarts the Dock (killall Dock)

Moving this to Infra & Ops as this seems to be an issue in the 10.14 setup rather than the test.

:dividehex - while we are discussing the topic of macosx1014 environment, this one appears to be related to the setup of the machines as well, at least according to :standard8. Rest of the logs from a recent run of mochitest-browser-chrome shows the following:

19:39:26     INFO - TEST-START | browser/components/shell/test/browser_1119088.js
19:39:26     INFO - TEST-INFO | started process screencapture
19:39:26     INFO - TEST-INFO | screencapture: exit 0
19:39:26     INFO - Buffered messages logged at 19:39:26
19:39:26     INFO - Entering test bound 
19:39:26     INFO - Buffered messages finished
19:39:26     INFO - TEST-UNEXPECTED-FAIL | browser/components/shell/test/browser_1119088.js | Desktop background database must exist for test to run. - 
19:39:26     INFO - Stack trace:
19:39:26     INFO - chrome://mochikit/content/browser-test.js:test_ok:1313
19:39:26     INFO - chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js:null:23
19:39:26     INFO - resource://testing-common/BrowserTestUtils.jsm:withNewTab:111
19:39:26     INFO - chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js:null:9
19:39:26     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1115
19:39:26     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1143
19:39:26     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:1004
19:39:26     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
19:39:26     INFO - Not taking screenshot here: see the one that was previously logged
19:39:26     INFO - TEST-UNEXPECTED-FAIL | browser/components/shell/test/browser_1119088.js | Uncaught exception - [Exception... "Component returned failure code: 0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST) [nsIFile.copyTo]"  nsresult: "0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST)"  location: "JS frame :: chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js :: <TOP_LEVEL> :: line 30"  data: no]
19:39:26     INFO - Stack trace:
19:39:26     INFO - @chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js:30:25
19:39:26     INFO - withNewTab@resource://testing-common/BrowserTestUtils.jsm:111:24
19:39:26     INFO - async*@chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js:9:26
19:39:26     INFO - Tester_execTest/<@chrome://mochikit/content/browser-test.js:1115:34
19:39:26     INFO - Tester_execTest@chrome://mochikit/content/browser-test.js:1143:12
19:39:26     INFO - nextTest/<@chrome://mochikit/content/browser-test.js:1004:14
19:39:26     INFO - SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:803:59
19:39:26     INFO - Leaving test bound 
19:39:26     INFO - GECKO(1792) | JavaScript error: resource://testing-common/PromiseTestUtils.jsm, line 110: uncaught exception: Object
19:39:26     INFO - GECKO(1792) | MEMORY STAT vsizeMaxContiguous not supported in this build configuration.
19:39:26     INFO - GECKO(1792) | MEMORY STAT | vsize 6975MB | residentFast 334MB | heapAllocated 158MB
19:39:26     INFO - TEST-OK | browser/components/shell/test/browser_1119088.js | took 351ms
19:39:26     INFO - Not taking screenshot here: see the one that was previously logged
19:39:26     INFO - TEST-UNEXPECTED-FAIL | browser/components/shell/test/browser_1119088.js | Found an unexpected tab at the end of test run: about:logo - 

It seems the test expects the database to be present prior to the start of the test.

:spohl, mconley - I see that you have worked on this test in the past. :dividehex has informed me (via email) that making this change may be difficult due to the privacy and security enhancement made by Apple in macosx1014. Would it be acceptable to have this test disabled for macosx1014, or would the preferred approach be to rewrite this test such that it can also run on macosx1014?

I'm not usually in favor of disabling tests, but I'm in favor of disabling this particular test on 10.14 and above. It was already an unusually aggressive test that intentionally killed the Dock to reload the previous Desktop image. We may just have to keep an eye out for user reports in case this feature breaks in the future.

:dividehex - thanks for the PR, I was under impression this test could not be reconciled with macosx1014 but hoping your patch would be the silver bullet!

(In reply to Edwin Gao (:egao) from comment #16)

:dividehex - thanks for the PR, I was under impression this test could not be reconciled with macosx1014 but hoping your patch would be the silver bullet!

I hope so too. I've deployed this new profile config to the 'gecko-t-osx-1014-staging' workers. Can you re-attempt this test against that workerType? If it works, I'll deploy it to 'gecko-t-osx-1014'

:dividehex - I tried running mochitest-browser-chrome against the macosx1014-staging workers:

https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=e9bddd4087b14513a1045aa2a35fcc212199fc2d

Looks like the test still fails, with the same error.

21:34:18     INFO - TEST-START | browser/components/shell/test/browser_1119088.js
21:34:18     INFO - TEST-INFO | started process screencapture
21:34:19     INFO - TEST-INFO | screencapture: exit 0
21:34:19     INFO - Buffered messages logged at 21:34:18
21:34:19     INFO - Entering test bound 
21:34:19     INFO - Buffered messages finished
21:34:19     INFO - TEST-UNEXPECTED-FAIL | browser/components/shell/test/browser_1119088.js | Desktop background database must exist for test to run. - 
21:34:19     INFO - Stack trace:
21:34:19     INFO - chrome://mochikit/content/browser-test.js:test_ok:1576
21:34:19     INFO - chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js:null:26
21:34:19     INFO - resource://testing-common/BrowserTestUtils.jsm:withNewTab:124
21:34:19     INFO - chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js:null:9
21:34:19     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1346
21:34:19     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1381
21:34:19     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:1209
21:34:19     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
21:34:19     INFO - Not taking screenshot here: see the one that was previously logged
21:34:19     INFO - TEST-UNEXPECTED-FAIL | browser/components/shell/test/browser_1119088.js | Uncaught exception - [Exception... "Component returned failure code: 0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST) [nsIFile.copyTo]"  nsresult: "0x80520006 (NS_ERROR_FILE_TARGET_DOES_NOT_EXIST)"  location: "JS frame :: chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js :: <TOP_LEVEL> :: line 35"  data: no]
21:34:19     INFO - Stack trace:
21:34:19     INFO - @chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js:35:27
21:34:19     INFO - withNewTab@resource://testing-common/BrowserTestUtils.jsm:124:24
21:34:19     INFO - async*@chrome://mochitests/content/browser/browser/components/shell/test/browser_1119088.js:9:26
21:34:19     INFO - Tester_execTest/<@chrome://mochikit/content/browser-test.js:1346:34
21:34:19     INFO - Tester_execTest@chrome://mochikit/content/browser-test.js:1381:11
21:34:19     INFO - nextTest/<@chrome://mochikit/content/browser-test.js:1209:14
21:34:19     INFO - SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:803:59
21:34:19     INFO - Leaving test bound 
21:34:19     INFO - GECKO(2122) | MEMORY STAT vsizeMaxContiguous not supported in this build configuration.
21:34:19     INFO - GECKO(2122) | MEMORY STAT | vsize 6956MB | residentFast 340MB | heapAllocated 152MB
21:34:19     INFO - TEST-OK | browser/components/shell/test/browser_1119088.js | took 355ms
21:34:19     INFO - Not taking screenshot here: see the one that was previously logged
21:34:19     INFO - TEST-UNEXPECTED-FAIL | browser/components/shell/test/browser_1119088.js | Found an unexpected tab at the end of test run: about:logo - 
21:34:19     INFO - checking window state
21:34:19     INFO - GECKO(2122) | JavaScript error: resource://testing-common/PromiseTestUtils.jsm, line 112: uncaught exception: Object

With this information, what next steps would you suggest? We have buy-in from :spohl about disabling this test, so perhaps it might just come to that.

(In reply to Edwin Gao (:egao) from comment #18)

With this information, what next steps would you suggest? We have buy-in from :spohl about disabling this test, so perhaps it might just come to that.

I don't have any other tricks up my sleeve to make this work so I'll leave its fate up to you.

:dividehex, I think we could rework this test to not depend on the private desktoppicture.db file. I have some python code that prints out the current desktop image path[1] using the desktopImageURLForScreen[2] API and we could use that to validate we can set the desktop background successfully. Would it be OK to take this bug?

1. https://gist.github.com/hafta/8efc203d33fea574959152111343a554
2. https://developer.apple.com/documentation/appkit/nsworkspace/1530635-desktopimageurlforscreen?language=objc

(In reply to Haik Aftandilian [:haik] from comment #20)

Would it be OK to take this bug?

Sure, go ahead. Thanks!

Refactor the test to use a python program to check that the background was successfully changed and re-enable the test on macOS.

For running in automation, this test is dependent on Mac test runner instances being configured to allow users to change the desktop background.

The posted patch (comment 23) has been tested locally. Once :dividehex's patch is live, we can test this on automation and move forward with code review.

(In reply to Haik Aftandilian [:haik] from comment #24)

The posted patch (comment 23) has been tested locally. Once :dividehex's patch is live, we can test this on automation and move forward with code review.

Profile change should be well propagated now. Go ahead and test.

(In reply to Jake Watkins [:dividehex] from comment #25)

(In reply to Haik Aftandilian [:haik] from comment #24)

The posted patch (comment 23) has been tested locally. Once :dividehex's patch is live, we can test this on automation and move forward with code review.

Profile change should be well propagated now. Go ahead and test.

Thanks!

(In reply to Jake Watkins [:dividehex] from comment #25)

(In reply to Haik Aftandilian [:haik] from comment #24)

The posted patch (comment 23) has been tested locally. Once :dividehex's patch is live, we can test this on automation and move forward with code review.

Profile change should be well propagated now. Go ahead and test.

Debugging on the loaner, the desktop seems to be locked as a gray background. I tried using macOS System Preferences to change the background and regardless of which image is selected, the background stays gray. Is there any information worth collecting on the machine to help debug this?

(In reply to Haik Aftandilian [:haik] from comment #27)

(In reply to Jake Watkins [:dividehex] from comment #25)

(In reply to Haik Aftandilian [:haik] from comment #24)

The posted patch (comment 23) has been tested locally. Once :dividehex's patch is live, we can test this on automation and move forward with code review.

Profile change should be well propagated now. Go ahead and test.

Debugging on the loaner, the desktop seems to be locked as a gray background. I tried using macOS System Preferences to change the background and regardless of which image is selected, the background stays gray. Is there any information worth collecting on the machine to help debug this?

I've logged into the loaner, #388, and I'll test/check also and update you with what I find.

I found that it is blocking changing the background (for me also -- to an image, standard or dynamic, or a color). I'll try modifying the profile to see if there is a change that allows changing the background.

:haik, I think we need to remove the profile and set a default background in another way to allow the background to be later changed by a task or Firefox. Does that sound like it will work for you? How is the task trying to change the background?

Although the profile attribute "locked" is false, the desktop background cannot be changed.

There are reports of the same problem on macos admin forums (https://www.jamf.com/jamf-nation/discussions/15469/setting-desktop-background-for-all-users-with-configuration-profiles)) and writeups like, https://scriptingosx.com/2018/09/managing-the-desktop-picture-on-macos/ : "With this configuration profile in place, the desktop picture is locked. The user can still open the Desktop preference pane but the selection will be ignored. You would need to be able to remove the profile to change the desktop picture again."
On the forums, like for jamf (https://www.jamf.com/jamf-nation/discussions/15469/setting-desktop-background-for-all-users-with-configuration-profiles), admins report solutions like a symlink/replace DefaultBackground.jpg, use applescript, or more recent mojave+ posts point to using https://github.com/scriptingosx/desktoppr

I created new profiles to test if the profile works (created with ProfileCreator like the howto at https://kb.parallels.com/en/124987):

1. as-is, with the locked attribute set to false
2. omitting the locked attribute

For each, I tested on my dev macos machine (no puppet that would reapply the profile). I removed existing wallpaper profiles, then applied the profile and rebooted, and the wallpaper could not be changed with either profile setting.

(In reply to Dave House [:dhouse] from comment #30)

:haik, I think we need to remove the profile and set a default background in another way to allow the background to be later changed by a task or Firefox. Does that sound like it will work for you? How is the task trying to change the background?

nevermind about "How", I read some of the backlog and saw those details are already discussed.
It might be worth looking at the desktoppr utility some macos admins report using: https://github.com/scriptingosx/desktoppr (though it looks like they're using the same api calls as the python script)

Thanks for looking into this.

A bit of an explanation about this test: we have code in Firefox that sets the desktop background using Apple's official API setDesktopImageURL. The test triggers this code to run to set the desktop background. The test also needs to make sure this works so it must read back the current desktop background and confirm we were able to set it. There is an official API to do that, but we don't use that in Firefox so instead of adding code to do this in Firefox only for the sake of testing, the test checks the desktop image using another way. The existing test used the undocumented desktoppicture.db file and I have posted patches to change the test to use a python script that calls the official API.

The desktoppr program uses the same API as Firefox (but it's written in Swift not Objective-C).

Do we have to have a profile for desktop background or could we have no profile for that so essentially we don't control it?

If this seems broken at the OS level and we just don't have a way to allow setting the desktop background on automation (without a lot of scripting work), we can close this as WONT-FIX for now and file a bug with Apple if that's the right place.

(In reply to Haik Aftandilian [:haik] from comment #32)

Thanks for looking into this.

A bit of an explanation about this test: we have code in Firefox that sets the desktop background using Apple's official API setDesktopImageURL. The test triggers this code to run to set the desktop background. The test also needs to make sure this works so it must read back the current desktop background and confirm we were able to set it. There is an official API to do that, but we don't use that in Firefox so instead of adding code to do this in Firefox only for the sake of testing, the test checks the desktop image using another way. The existing test used the undocumented desktoppicture.db file and I have posted patches to change the test to use a python script that calls the official API.

The desktoppr program uses the same API as Firefox (but it's written in Swift not Objective-C).

Do we have to have a profile for desktop background or could we have no profile for that so essentially we don't control it?

If this seems broken at the OS level and we just don't have a way to allow setting the desktop background on automation (without a lot of scripting work), we can close this as WONT-FIX for now and file a bug with Apple if that's the right place.

Jake, what's your opinion? From what I found researching and testing, we need to use something other than a profile to set and ensure the background is set to a default at reboot but allow for it to be changed by the user.

Haik, I think we can do the same thing in automation setup as the test. We can remove the mobileconfig/profile and use something like a small python script calling the same setDesktopImageURL to reset it at boot.

I question why we need to be enforcing the desktop background at all. It seems like something that should be controlled entirely by the task at hand. I would suggest we remove the desktop background profile all together. Has this been tried on the loaner?

(In reply to Jake Watkins [:dividehex] from comment #34)

I question why we need to be enforcing the desktop background at all. It seems like something that should be controlled entirely by the task at hand. I would suggest we remove the desktop background profile all together. Has this been tried on the loaner?

Removing the profile sounds good to me. And we could try setting the background to something drastically different to see if it causes test failures (I am concerned there may be tests with fullscreen screenshots or of ui elements tinted by mojave to match the desktop).

Hi Haik. To avoid breaking other tests, I'm thinking to remove the background-lock profile and copy the mac_desktop_image.py to be executed at each boot to set the background. Does that plan work for you?

(In reply to Dave House [:dhouse] from comment #36)

Hi Haik. To avoid breaking other tests, I'm thinking to remove the background-lock profile and copy the mac_desktop_image.py to be executed at each boot to set the background. Does that plan work for you?

Yep, sounds good.

https://hg.mozilla.org/mozilla-central/rev/18e6a640ccfe