Closed
Bug 1359288
Opened 8 years ago
Closed 7 years ago
Every Android jsreftest run, green or not, includes a "REFTEST ERROR | Force stop because we keep running into test "None""
Categories
(Testing :: Reftest, enhancement)
Tracking
(firefox55 fixed)
RESOLVED
FIXED
mozilla55
Tracking | Status | |
---|---|---|
firefox55 | --- | fixed |
People
(Reporter: philor, Assigned: shinglyu)
References
Details
Attachments
(1 file)
Okay, strictly speaking I don't know about "every" since I only checked a random selection of the 40 opt and 100 debug chunks, but every green run I checked looked like https://treeherder.mozilla.org/logviewer.html#?job_id=93767161&repo=mozilla-inbound - particularly bad in the way I first found out about it, with orange runs which had an actual test failure which failed to produce any sane parsable output, so that the "REFTEST ERROR | Force stop because we keep running into test "None"" was the only thing that Treeherder thought had gone wrong.
Comment 1•8 years ago
|
||
I've seen it as well.
Assignee | ||
Comment 3•8 years ago
|
||
Sorry for the confusion. In bug 1344991 I made reftest able to recover from a crash, but to prevent the test from re-running a crashed test forever, I added the fuse mechanism that produces this error. The root cause is that I only enable the recovery feature on desktop, but used a stub for Android.
The best solution is to add the crash recovery feature to android as well, but I can't estimate how long it will take. If this is very urgent I can suppress the error on Android platform.
Flags: needinfo?(shing.lyu)
Comment 4•8 years ago
|
||
I would say that if the error can be suppressed without causing negative effects for android then you should do it immediately. If not, then the original patch should be backed out until this is fixed.
Comment hidden (mozreview-request) |
Assignee | ||
Comment 6•8 years ago
|
||
Bob,
The fix was simpler then I expected. So I not pushed a patch that will fix this permanently.
Dbaron,
Would you mind reviewing this for me? This was a followup from the previous resume-from-crash reftest feature. But I didn't do the Android part because I'm not familiar with the code for Android back then. Now I fixed it and it will help reduce the noise for Android try server log. Thanks!
Assignee | ||
Comment 7•8 years ago
|
||
Here is a try run without the errors: https://treeherder.mozilla.org/#/jobs?repo=try&revision=78a285dadd1def3b8f01426d76bda90c0ede96fe
Updated•8 years ago
|
Attachment #8866153 -
Flags: review?(dbaron) → review?(jmaher)
Comment 8•8 years ago
|
||
mozreview-review |
Comment on attachment 8866153 [details]
Bug 1359288 - Enable Android reftest to resume from crashes.
https://reviewboard.mozilla.org/r/137760/#review140988
thanks, I agree this looks simple
Attachment #8866153 -
Flags: review?(jmaher) → review+
Pushed by slyu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/66513d27bc96
Enable Android reftest to resume from crashes. r=jmaher
Comment 10•8 years ago
|
||
Backed out for throwing exceptions during Android robocop tests:
https://hg.mozilla.org/integration/autoland/rev/cede71157daf70061897b0d5d8ef723f48bf7620
Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=66513d27bc96ec905604b45056ba0a40b622e4c2&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable&filter-resultStatus=success&filter-searchStr=robocop
Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=98631969&repo=autoland
[task 2017-05-12T05:46:17.425939Z] 05:46:17 INFO - SUITE-START | Running 14 tests
[task 2017-05-12T05:50:37.236250Z] 05:50:37 INFO - INFO | automation.py | Application pid: 0
[task 2017-05-12T05:50:37.573622Z] 05:50:37 INFO - SimpleTest START
[task 2017-05-12T05:50:37.574364Z] 05:50:37 INFO - TEST-START | testBrowserDiscovery
[task 2017-05-12T05:50:37.574453Z] 05:50:37 INFO - GECKO | EventExpecter: no longer listening for Gecko:Ready
[task 2017-05-12T05:50:37.589927Z] 05:50:37 INFO - GECKO | EventExpecter: no longer listening for Robocop:Java
[task 2017-05-12T05:50:37.590006Z] 05:50:37 INFO - TEST-OK | testBrowserDiscovery | took 74596ms
[task 2017-05-12T05:50:37.590734Z] 05:50:37 INFO - TEST-START | Shutdown
[task 2017-05-12T05:50:37.591117Z] 05:50:37 INFO - Passed: 201
[task 2017-05-12T05:50:37.591809Z] 05:50:37 INFO - Failed: 0
[task 2017-05-12T05:50:37.591865Z] 05:50:37 INFO - Todo: 0
[task 2017-05-12T05:50:37.592047Z] 05:50:37 INFO - SimpleTest FINISHED
[task 2017-05-12T05:50:59.054270Z] 05:50:59 INFO - INFO | automation.py | Application ran for: 0:04:40.645413
[task 2017-05-12T05:50:59.054526Z] 05:50:59 INFO - INFO | zombiecheck | Reading PID log: /tmp/tmpCCSpIFpidlog
[task 2017-05-12T05:50:59.643946Z] 05:50:59 INFO - /data/tombstones does not exist; tombstone check skipped
[task 2017-05-12T05:51:00.355487Z] 05:51:00 INFO - 0 ERROR Automation Error: Exception caught while running tests
[task 2017-05-12T05:51:00.355781Z] 05:51:00 INFO - Traceback (most recent call last):
[task 2017-05-12T05:51:00.356080Z] 05:51:00 INFO - File "/home/worker/workspace/build/tests/mochitest/runrobocop.py", line 465, in runSingleTest
[task 2017-05-12T05:51:00.356729Z] 05:51:00 INFO - self.log.debug("runApp completes with status %d" % result)
[task 2017-05-12T05:51:00.356835Z] 05:51:00 INFO - TypeError: not all arguments converted during string formatting
[task 2017-05-12T05:54:48.370052Z] 05:54:48 INFO - INFO | automation.py | Application pid: 0
Flags: needinfo?(shing.lyu)
Comment hidden (mozreview-request) |
Assignee | ||
Comment 12•8 years ago
|
||
Comment on attachment 8866153 [details]
Bug 1359288 - Enable Android reftest to resume from crashes.
I fixed the reason for the autophone failures. Autophone mochitest and robocop also inherits the runApp() API, so I change them to fit the reftest runApp() API, but ignoring the second argument to keep the behavior the same.
Try result:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=ef4a93d3fccd0a93a9addd8f8047788f1b86522a
There are some Mdm1 fails but they also appear in other m-c pushes, so looks like intermittents.
Flags: needinfo?(shing.lyu)
Attachment #8866153 -
Flags: review+ → review?(jmaher)
Comment 13•8 years ago
|
||
mozreview-review |
Comment on attachment 8866153 [details]
Bug 1359288 - Enable Android reftest to resume from crashes.
https://reviewboard.mozilla.org/r/137760/#review142982
thanks for the update!
Attachment #8866153 -
Flags: review?(jmaher) → review+
Assignee | ||
Comment 14•7 years ago
|
||
Hi Bob,
We have some gl* errors here, are they known issues for Autophone test? I didn't see them being run very often. Do you think this patch is OK to land? Is there any other android tests that might be affected?
https://treeherder.mozilla.org/#/jobs?repo=try&revision=71324e70156f6ba9cf6b8b98740953c215b7b7a9
Flags: needinfo?(bob)
Comment 15•7 years ago
|
||
The gl conf tests are mochitests and shouldn't be affected by reftest only changes. The failures for the gl tests are already filed under Bug 1359078. The Cdm{1..4}, Rov and Rwv tests are reftest based and are green on your try.
Flags: needinfo?(bob)
Comment 16•7 years ago
|
||
Pushed by slyu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3241038a0214
Enable Android reftest to resume from crashes. r=jmaher
Comment 17•7 years ago
|
||
bugherder |
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox55:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
Updated•6 years ago
|
Assignee: nobody → shing.lyu
You need to log in
before you can comment on or make changes to this bug.
Description
•