I am not sure what landed in the last 24 hours, but I am not able to run |mach raptor-test -t raptor-wasm-misc| anymore which I was developing. $ ./mach raptor-test -t raptor-wasm-misc Updating external benchmarks from https://github.com/mozilla/perf-automation Cloning into 'c:/Users/elvis/.mozbuild\performance-tests'... remote: Counting objects: 281, done. remote: Compressing objects: 100% (20/20), done. remote: Total 281 (delta 6), reused 15 (delta 2), pack-reused 258 Receiving objects: 100% (281/281), 40.68 MiB | 2.75 MiB/s, done. Resolving deltas: 100% (18/18), done. Checking out files: 100% (254/254), done. Note: checking out '6beb3d3e22abce8cf8e2e89bc45acd4152258f12'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b <new-branch-name> HEAD is now at 6beb3d3 Use window.postMessage to send results back to Raptor's control server [Error 2] The system cannot find the file specified: u'c:\\Users\\elvis\\mozilla-inbound\\objdir-frontend\\testing\\raptor\\benchmarks\\ARES-6' Possibly we need to adjust our cloning?

here is what I have in the performance-benchmarks repo locally: $ ls ~/.mozbuild/performance-tests/ LICENSE benchmarks pagesets elvis@JMAHER-WIN10 ~/mozilla-inbound $ ls ~/.mozbuild/performance-tests/benchmarks/ unity-webgl this is odd since I recently added code to performance-tests for the web-tooling-benchmark, that seems to have disappeared.

backing out bug 1472800 locally seemed to solve the above mentioned problem, but then when I run benchmarks it appears that I cannot copy data from third_part/webkit/PerformanceTests -> objdir/testing/raptor/benchmarks; so something odd is going on.

:ahal, do you have any additional data you could provide here?

That bug pins the repository to a revision (to make sure everyone is running the same version of the benchmarks locally): https://searchfox.org/mozilla-central/source/testing/raptor/mach_commands.py#25 You'll just need to update that to something that includes your changes. One gotcha to this sytem is that if the new revision contains unrelated changes to other benchmarks (for example if I had changed unity-webgl but then not created a new github release for those changes), doing this would update unity-webgl for people locally to a version newer than the one running in CI. This probably isn't a huge deal for now, but we may want to make it better in the future.

I think the problem is that we have benchmarks in third_party/webkit/PerformanceTests which are not getting copied over- ares-6 is not in the github repo.

ares-6 is a jsshell-bench test so shouldn't have anything to do with raptor or the perf-automation repo. Do you still have problems after updating the revision I linked to in comment 4? Fwiw I can run raptor benchmarks that live in both ~/.mozbuild/performance-tests and third_party/webkit/PerformanceTests just fine.

I get this same error on any benchmark I try to run even if it lives in third_party/ in m-c.

even rebooting my computer and working on a fresh branch didn't seem to help. Basically right now I cannot run |mach raptor-test|. I am on windows 10- otherwise I don't know of anything odd or custom I am doing.

This is a bug in how I was using the FileCopier.

This fixes a bug where the benchmarks weren't being copied properly on Windows. While it is slightly slower to add every file to the manifest (~0.2 seconds), it still outperforms shutil.copytree and is more robust (will use links on Linux/Mac while using copy on Windows).

Comment on attachment 8999699 [details] Bug 1482057 - [raptor] Fix benchmark setup on Windows, r=jmaher Joel Maher ( :jmaher ) (UTC+2) has approved the revision.

Pushed by ahalberstadt@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/f88a04dbc4b1 [raptor] Fix benchmark setup on Windows, r=jmaher