Filed by: bclary [at] mozilla.com https://treeherder.mozilla.org/logviewer.html#?job_id=148552619&repo=autoland https://autophone.s3.amazonaws.com/v1/task/dO1oxfltQzaQGGMWV1oCDw/runs/0/artifacts/public/build/b7aae30f-96db-44e5-883a-0f586c032546-autophone.log

There have been 40 failures in the last 2 days. This fails only on opt build type. Occurrences per platform: - 34 on android-6-0-armv8-api16 - 5 on android-4-4-armv7-api16 - 1 on windows10-64 This is a recent log file: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=148964096&lineNumber=902 And a snippet with the failure: 2017-12-01 01:50:35,659 nexus-6p-05 INFO UnitTestJob mozilla-central 20171130233148 opt api-23 android-api-16 mochitest-dom-media-tests 339 INFO Adding ICE candidate: {"candidate":"candidate:3 1 TCP 2105524479 2620:101:80fc:120:31ee:9756:bb08:8d68 9 typ host tcptype active","sdpMid":"sdparta_0","sdpMLineIndex":0} 900 2017-12-01 01:50:35,659 nexus-6p-05 INFO UnitTestJob mozilla-central 20171130233148 opt api-23 android-api-16 mochitest-dom-media-tests 340 INFO Adding ICE candidate: null 901 2017-12-01 01:50:35,659 nexus-6p-05 INFO UnitTestJob mozilla-central 20171130233148 opt api-23 android-api-16 mochitest-dom-media-tests Buffered messages finished 902 2017-12-01 01:50:35,659 nexus-6p-05 INFO UnitTestJob mozilla-central 20171130233148 opt api-23 android-api-16 mochitest-dom-media-tests 341 INFO TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_transceivers.html | Test timed out. 903 2017-12-01 01:50:35,659 nexus-6p-05 INFO UnitTestJob mozilla-central 20171130233148 opt api-23 android-api-16 mochitest-dom-media-tests reportError@SimpleTest/TestRunner.js:121:7 904 2017-12-01 01:50:35,659 nexus-6p-05 INFO UnitTestJob mozilla-central 20171130233148 opt api-23 android-api-16 mochitest-dom-media-tests TestRunner._checkForHangs@SimpleTest/TestRunner.js:142:7

Byron, it looks like there were some time-outs in autophone tests since you landed 'test_peerConnection_transceivers.html' last week. Could you please take a look? Thanks a lot.

I think I know what is happening here.

Comment on attachment 8934280 [details] Bug 1421775: Await negotiationneeded before SLD for the other PC so we don't miss it. https://reviewboard.mozilla.org/r/205192/#review210770

Under normal circumstances, I would try to elicit the failure with lots of retriggers, to check that the fix worked. However, since autophone is running on actual hardware in a lab, I'm just going to land this now. We can re-open if the fix doesn't work.

Pushed by bcampen@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/7cc99d649d75 Await negotiationneeded before SLD for the other PC so we don't miss it. r=jib

Ok, this does not look fixed.

Comment on attachment 8935458 [details] Bug 1421775 - Attempt 2: Don't try to update negotiationneeded until after the stuff we do in parallel with sRD/sLD finishes. https://reviewboard.mozilla.org/r/206350/#review212116

Pushed by bcampen@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/a61bb7e683d9 Attempt 2: Don't try to update negotiationneeded until after the stuff we do in parallel with sRD/sLD finishes. r=jib

This seems to have fixed itself somehow...

Ok, so it definitely stopped happening on autophone on mozilla-inbound after the landing of Attempt 2. The remaining instances seem to be some other problem.

So the remaining occurrences of this bug seem to be timeouts that happen _after_ TEST-OK has happened, and then warnings about finish() being called multiple times...

This seems to have stopped happening on the 13th. Not sure why. I guess we can keep an eye on it.