To start, sorry for the rather abrupt disabling here :( After last week's NSS chemspill, we noticed frequent timeouts of the toolkit webapps tests. At the time, they appeared to be limited to Gecko33 and under. Given the unlikeliness of them getting attention on older branches, I skipped the tests on those branches with Fabrice's blessing. Fast-forward to today where something made them go crazy on inbound as well :( https://treeherder.mozilla.org/ui/logviewer.html#?job_id=2712798&repo=mozilla-inbound The log above is a very typical instance of this failure. It happens only on OSX and affects both opt and debug. Due to the failure rate, I need to disable them right away. Sorry for the lack of notice :(

Looks like the error is: "webapprt[1363:303] got exception: Cannot locate binary for this App", I wonder if bug 1075492 fixed this problem too.

(In reply to Marco Castelluccio [:marco] from comment #2) > Looks like the error is: "webapprt[1363:303] got exception: Cannot locate > binary for this App", I wonder if bug 1075492 fixed this problem too. No. These were happening on inbound right up to the push prior to the disabling landing on it.

Are you able to tell when these tests started going crazy?

Oh, I bet bug 1075492 is actually the culprit here, sorry. Patch coming up.

I'm not quite sure how to test this, but this should fix the reported failures.

A Try run with comment 1 reverted and 5-10 retriggers should make it pretty clear if your patch works or not.

Comment on attachment 8499101 [details] [diff] [review] Patch RyanVM said that we see strange errors on other branches as well, where my patch from bug 1075492 hasn't (shouldn't?) have landed yet. I'll take this opportunity to improve this code.

We're now checking whether the path to Contents/MacOS/webapprt exists on disk rather than asking the OS to provide us with a path to the webapprt directory. The API that was previously used (NSBundle pathForAuxiliaryExecutable) is intended to retrieve helpers and applications under Contents/MacOS in the app bundle. The webapprt folder obviously isn't a helper or an application bundle. The API just happened to return the proper path because the OS treats any folder under Contents/MacOS as an application bundle. Reverted comment 1 and submitted to try: https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=bafa75eaaeb2

I just triggered a few more m-oth runs to confirm the fix. All completed runs have been green so far: https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=bafa75eaaeb2

FWIW, they're still disabled on 10.6 ;)

Hah, that's awesome! Way to get my hopes up. :-)

I pushed this to try again without the disabling of 10.6 https://tbpl.mozilla.org/?tree=Try&rev=187951ba0a24 https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=187951ba0a24

I triggered a few more 10.6 m-oth runs. Since 10.8 is lagging and the previous push had 10.8 enabled I am not going to trigger additional 10.8 jobs on my push.

It doesn't seem to have fixed the 10.6 errors that started before v2 signing landed and is covered by bug 993690. Let's go with the 10.8 fix that was caused by the v2 signing and let bug 993690 handle the 10.6 failures.

Since this is a straight backout, I'm not sure it has to be reviewed again. Ryan, should I land this at the same time as the patch in this bug? Or are there other reasons why the tests should remain disabled? Thanks!

Comment on attachment 8499500 [details] [diff] [review] Reenable tests Just include it with the patch.

Folded backout into this patch. Carrying over r+. Waiting for inbound to reopen. Landed on oak: https://hg.mozilla.org/projects/oak/rev/ebf3623173cf

Landed on aurora in the Mac V2 signing combined patch in bug 1047584