The mozperftests in Web QA's Jenkins instance are failing due to bug 915156 but taking up valuable resources by running regularly against physical devices. I've disabled them for now, and they can be re-enabled once the blocker is fixed.

Dave, Can you re-enable these now that Hub's landed bug 915156? Hub, Does bug 947259 or anything stop Dave from re-enabling mozperftest jobs? Thanks, Mike

I don't think it prevents from doing it. That's where we'll know :-D But they will take more time to run than needed.

I just tried (after installing node and npm on the machines) and had a lot of timeout exceptions like "Error: timeout of 10000ms exceeded". If you're connected to the VPN you can access the console log here: http://qa-selenium.mv.mozilla.com:8080/view/B2G%20Perf/job/b2g.hamachi.mozilla-central.master.mozperftest/634/console I also noticed that there's no perf.json file to be submitted to datazilla. I ran these tests locally and didn't see the timeouts, but also still did not see a perf.json file.

Unfortunately I don't have VPN :-/

Console log from failed Jenkins build.

I see you do |adb forward tcp:2828 tcp:2828| before running |make test-perf|. The runner is supposed to take care of that (using a randomized port). What if you remove it.... (I don't know how the adb forward stuff deal with setting a forward from another port to the same target when there is already one set).

I've just triggered a run without it, but are you saying that |make test-perf| somehow changes the port that Marionette is listening on the device?

No we just use a random port for the client on the PC host and set the proper adb forward to tcp:2828 on the device.

This is still failing without the adb forward in the same way. When I run the |make test-perf| locally I get the desktop build downloaded and it runs against that. Is it possible that Jenkins is trying to do this instead of running on device? If so, how do I force it to use the device?

See: https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_performance_tests#Requirement By default it runs on desktop b2g. I didn't notice in the log.

Okay, so with that change we get some results but still lots of issues. I've attached the latest console log, some issues I'm seeing are mentioned below: 1. ./bin/gaia-perf-marionette: line 20: >: command not found 2. stderr: error: cannot bind socket 3. Error: timeout of 10000ms exceeded 4. TypeError: Cannot call method \\'stop\\' of undefined 5. StaleElementReference 6. NoSuchElement I also still don't see a JSON file for submission to DataZilla. How can I generate this, or is there now a way to submit directly to DataZilla? Do you want me to raise these are separate issues?

1. This is harmless. I have a patch for that, and I should file a bug about it. 2. Looks like you can't connect. Ooops. 3. 4. Possibly caused by 2. 5. 6. Likely the same. I'll dig a bit more to see if I don't find something else.

I can replicate the 'stderr: error: cannot bind socket' issue locally on my Hamachi. This also appears in the stdout and therefore the perf.json file when running as: |make -s test-perf > perf.json| and preventing us from publishing the results to DataZilla.

Just out of curiosity, what do you use locally to run it?

(ie what OS)

(In reply to Dave Hunt (:davehunt) from comment #13) > I can replicate the 'stderr: error: cannot bind socket' issue locally on my > Hamachi. This also appears in the stdout and therefore the perf.json file > when running as: |make -s test-perf > perf.json| and preventing us from > publishing the results to DataZilla. Same for me on a hamachi, my host os is a debian testing.

Fabrice, just out of curiosity, is adb in your path by default? I'm wondering is that's not the problem you are all having. BTW I found out that the test fail on Hamachi for some reason. See bug 949209. (I run it all on inari which is so much easier to flash)

(In reply to Hubert Figuiere [:hub] from comment #15) > (ie what OS) I'm on Mac, the Jenkins node is on Ubuntu. adb is in the path, and I do see some results. If adb was an issue I would expect no tests to run.

(In reply to Hubert Figuiere [:hub] from comment #17) > Fabrice, just out of curiosity, is adb in your path by default? I'm > wondering is that's not the problem you are all having. Yes, adb is in my path.

Fabrice, the fact that you can't run it locally on your machine is puzzling. :-/

The job in Jenkins now times out after three hours. I have attached the console log and disabled the job again.

Done.