Comment on attachment 8990962 [details] [diff] [review] Patch v1 Review of attachment 8990962 [details] [diff] [review]: ----------------------------------------------------------------- Makes sense. Do you have an example of reporting in a try run? Thanks for the patch. ::: js/src/tests/lib/results.py @@ +112,5 @@ > else: > test_result = (cls.FAIL, test.message) > result = cls.FAIL > + test_output += "expected %s, found %s" % (expected, test.status) > + if test.message: Is message always guaranteed to exist, even if it's None/empty string? Otherwise, hasattr should be tested for here.

Thanks! (In reply to Benjamin Bouvier [:bbouvier] from comment #1) > Comment on attachment 8990962 [details] [diff] [review] > Patch v1 > > Review of attachment 8990962 [details] [diff] [review]: > ----------------------------------------------------------------- > > Makes sense. Do you have an example of reporting in a try run? Thanks for > the patch. No, but locally it looks like this: REGRESSION - wasm/jsapi/interface.any.js [0|2|0|0] 40% =====================> | 0.5s ## wasm/jsapi/interface.any.js: rc = 0, run time = 0.087217 Harness status: OK (Reported by harness: None) Subtest "WebAssembly: property descriptor": expected FAIL, found PASS Subtest "WebAssembly: constructing": as expected: PASS Subtest "WebAssembly.validate": expected PASS, found FAIL (with message: "assert_true: enumerable expected true got false") > ::: js/src/tests/lib/results.py > @@ +112,5 @@ > > else: > > test_result = (cls.FAIL, test.message) > > result = cls.FAIL > > + test_output += "expected %s, found %s" % (expected, test.status) > > + if test.message: > > Is message always guaranteed to exist, even if it's None/empty string? > Otherwise, hasattr should be tested for here. wpttest.SubtestResult.__init__ does always set it, yes.

Pushed by Ms2ger@gmail.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/d8c824521e7b Improve output for failing wpt tests in jstests harness; r=bbouvier

Backed out 2 changesets (bug 1474569, bug 1474581) for wpt failures in test harness output on a CLOSED TREE Problematic push: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=b39da1b094e9f89a42ce93f305d4cf1c09c8b4a5&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-classifiedState=unclassified Failure: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&fromchange=383714d5acad1c17adb25b8145f0cf1c25f1261c&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-classifiedState=unclassified&selectedJob=187627918 Backout: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=25915c35842591babf7cfe5fc8ff0d5c3adbb327&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-classifiedState=unclassified Log: https://treeherder.mozilla.org/logviewer.html#?job_id=187627918&repo=mozilla-inbound

Pushed by Ms2ger@gmail.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/9d11c83f1d68 Improve output for failing wpt tests in jstests harness; r=bbouvier