While attempting an initial test task on the gecko-t-osx-1014 workers in mdc1, tasks failed when attempting to copy files located under /System. This function happens when setting up a virtualenv folder. I suspect this is related to the TCC and PCCC restriction within Mojave.

First, mozharness (and all tests within the task) should be using the installed python2 and python3 which are explicitly installed and located under /usr/local instead of the System provided python env under /System.

When mozharness copies the python site via shutil.copytree, it should also use the location non-system python.

I think we just need to explicitly set the python path somewhere, not sure where though.

https://searchfox.org/mozilla-central/source/third_party/python/virtualenv/virtualenv.py#330

https://tools.taskcluster.net/groups/WQiaEkdCSkmeapZxRFnf8A/tasks/EWjG_OnSTtuSixMqtOIU5Q

:jmaher pointed me at :ahal and :callek to help with this issue. Thoughts?

I'm not very familiar with how mozharness manages python/virtualenvs, but the code for that stuff is here:
https://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/base/python.py

Though, the python that gets used likely needs to be set in task configuration (i.e whatever python gets used to run the mozharness script). E.g in your log I see:

Executing command 0: 'python2.7' -u 'mozharness/scripts/desktop_unittest.py' --cfg 'mozharness/configs/unittests/mac_unittest.py' --mochitest-suite=chrome --installer-url 'https://queue.taskcluster.net/v1/task/fQ_7TAlZQnuVUdstX4P8lw/artifacts/public/build/target.dmg' --test-packages-url 'https://queue.taskcluster.net/v1/task/fQ_7TAlZQnuVUdstX4P8lw/artifacts/public/build/target.test_packages.json' --download-symbols ondemand --mochitest-suite=chrome --total-chunk=3 --this-chunk=1

Rather than hardcode python2.7, we can pass the full path to the proper python here:
https://searchfox.org/mozilla-central/source/taskcluster/taskgraph/transforms/job/mozharness_test.py#267

thanks for the tip ahal, I have data now:
https://treeherder.mozilla.org/#/jobs?repo=try&searchStr=test-mac&revision=96a96f509d98f90ce3f8e35990df75153755b877