Filed by: archaeopteryx [at] coole-files.de https://treeherder.mozilla.org/logviewer.html#?job_id=125837497&repo=autoland https://queue.taskcluster.net/v1/task/KR88d5PjSAifmIAKq3E3fw/runs/0/artifacts/public/logs/live_backing.log 09:31:52 INFO - 1538 INFO TEST-PASS | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | screen global indicator attribute as expected - 09:31:52 INFO - 1539 INFO TEST-PASS | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | only one global indicator window - 09:31:52 INFO - 1540 INFO requesting devices 09:31:52 INFO - Buffered messages finished 09:31:52 ERROR - 1541 INFO TEST-UNEXPECTED-FAIL | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | Test timed out -

There are 33 failures in the last 7 days, all on Windows-10-64 (enabled and disabled) all on debug build type and one on opt. Here is a most recent log example: https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=142293228&lineNumber=16191 :drno Could you please take a look?

Cubeb assertion on Windows only. Alex can you have a look and dispatch to other cubeb experts as needed? Thanks

This has raised to 40 failures in the last 7 days and besides the failures on windows10-64 we also got one on linux 64. Here's a part of the linux log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=144527472&lineNumber=3388 [task 2017-11-14T08:14:17.867Z] 08:14:17 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | Test timed out - [task 2017-11-14T08:14:17.867Z] 08:14:17 INFO - GECKO(1644) | MEMORY STAT | vsize 2525MB | residentFast 296MB | heapAllocated 108MB [task 2017-11-14T08:14:17.868Z] 08:14:17 INFO - TEST-OK | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | took 90275ms [task 2017-11-14T08:14:17.868Z] 08:14:17 INFO - Not taking screenshot here: see the one that was previously logged [task 2017-11-14T08:14:17.869Z] 08:14:17 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | Found a tab after previous test timed out: https://example.com/browser/browser/base/content/test/webrtc/get_user_media.html - [task 2017-11-14T08:14:17.869Z] 08:14:17 INFO - GECKO(1644) | ++DOCSHELL 0x7fade0076800 == 1 [pid = 1732] [id = {eae72678-5f85-4433-bed0-55630cfc463c}] [task 2017-11-14T08:14:17.870Z] 08:14:17 INFO - GECKO(1644) | ++DOMWINDOW == 1 (0x7fade0088000) [pid = 1732] [serial = 5] [outer = (nil)] [task 2017-11-14T08:14:17.870Z] 08:14:17 INFO - GECKO(1644) | ++DOMWINDOW == 2 (0x7fade4783800) [pid = 1732] [serial = 6] [outer = 0x7fade0088000] [task 2017-11-14T08:14:17.871Z] 08:14:17 INFO - checking window state :drno or :achronop could you please take a closer look at this? Thank you.

This is timing out consistently in Windows coverage builds. Perhaps it's just a matter of increasing the timeout for the test?

Paul can you have a look, the usual assert on resampler: Assertion failed: out_len == output_frame_count, file z:\build\build\src\media\libcubeb\src\cubeb_resampler_internal.h, line 256 The number of frames in the buffer must be less than expected. Do we add silence in that case in windows?

The Linux case is not the same, the code path in Linux is not similar to windows. In linux, the error is something irrelevant to cubeb. We need a separate bug for that. Error in Linux: [task 2017-11-14T08:14:17.869Z] 08:14:17 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | Found a tab after previous test timed out: https://example.com/browser/browser/base/content/test/webrtc/get_user_media.html -

No I don't have time right now. But we do add silence.

Since Paul is busy I will have an initial investigation.

Have a look at the code that removes some data in the buffers in the resampling code.

I pushed a try with cubeb:4,MediaStreamGraph:4, you can find the results here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=55e284964486a100549e9e5bedc0f9e458f5d362 The error is not reproducible in this specific run but from the logs I got the cubeb stream configuration: 22:43:53 INFO - GECKO(9196) | [CubebOperation #1]: E/cubeb z:/build/build/src/media/libcubeb/src/cubeb_wasapi.cpp:1646: (000001A40045DA60) Setup capture: device=000001A405EE6E40 22:43:53 INFO - GECKO(9196) | [CubebOperation #1]: E/cubeb z:/build/build/src/media/libcubeb/src/cubeb_wasapi.cpp:1587: Setup requested=[f=2 r=48000 c=2 l=undefined] mix=[f=2 r=44100 c=2 l=stereo] 22:43:53 INFO - GECKO(9196) | [CubebOperation #1]: E/cubeb z:/build/build/src/media/libcubeb/src/cubeb_wasapi.cpp:1679: (000001A40045DA60) Setup render: device=0000000000000000 22:43:53 INFO - GECKO(9196) | [CubebOperation #1]: E/cubeb z:/build/build/src/media/libcubeb/src/cubeb_wasapi.cpp:1587: Setup requested=[f=2 r=48000 c=2 l=undefined] mix=[f=2 r=48000 c=2 l=stereo] 22:43:53 INFO - GECKO(9196) | [CubebOperation #1]: E/cubeb z:/build/build/src/media/libcubeb/src/cubeb_wasapi.cpp:1730: Target sample rate: 48000 According to the above the render does not need any resampling. Capture need to be resampled from 44100 to 48000. On a Debug build, the typical initial silence in the input buffer is 1928 frames (43.7ms) and in every callback we consume 448 frames. Resampler start dropping frames when buffer size worth 100ms of data, thus I do not believe that dropping is causing the problem here. Given that all the failures exist on Debug builds (link from comment 22) I believe that we hit the assert because we run out of input frames probably due to input underruns. Right now we initialize input buffers with 4 buffers of silence. I would suggest to increase that to 5 or 6 and see the result. Please let me know your thoughts.

Alex, it is apparently possible to reproduce the test failure consistently in a Windows coverage build. If you give me a patch to increase the buffers of silence, I can push it to try and see if it fixes the failure.

That's perfect thanks, I'll post the patch soon.

Marco, thanks for testing. This patch increases the initial input silence from a factor of 4 to a factor of 6. If that one does not work can we activate some logs in your try run? I will provide the patch for that.

We found the consistent failure in the ccov build is likely unrelated, as the assertion is missing there. In the ccov case we probably just need to increase the timeout.

It should be a good bandaid yeah.

Fix pushed for review on cubeb repo: https://github.com/kinetiknz/cubeb/pull/392

New failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=174541199&repo=autoland