Closed Bug 886381 Opened 11 years ago Closed 9 years ago

WebAudio static with closely timed AudioBufferSourceNodes

Categories

(Core :: Web Audio, defect, P2)

x86
macOS
defect

Tracking

()

RESOLVED WORKSFORME

People

(Reporter: inolen, Unassigned)

References

Details

(Whiteboard: [blocking-webaudio-])

Attachments

(2 files, 1 obsolete file)

I'd recently been working to emulate OpenAL's buffer queueing APIs with emscripten (https://github.com/kripken/emscripten/pull/1136#issuecomment-19887047). To do so, each buffer is being treated as an individual AudioBufferSourceNode and through some hackery they're being timed to play back to back with AudioBufferSourceNode::start(when). In Chrome, this has actually worked out great, however there's a terrible amount of clicking / popping noises in FF currently. I've setup two examples: http://www.inolen.com/openal_tiny_buffers/page.html and http://www.inolen.com/openal_big_buffers/page.html For reference, in the tiny buffers build, the buffers are configured as 4 buffers at 1470 bytes a piece (audio is at 44100 Hz, 16 bits, 1 channel) and in the large buffers build they are 4 buffers at 1470*30 bytes a piece. I haven't yet had time to extract this demo outside of the emscripten context. If that's required please let me know, otherwise if whoever is working on this is familiar with emscripten, you can grab the branch containing the OpenAL work here: https://github.com/inolen/emscripten/tree/incoming and there is a "test" in that branch which will produce the above output which can be invoked by running: python tests/runner.py browser.test_openal_buffers
Could be related to bug 883844.
Blocks: webaudio
Depends on: 883844
I expect the result would be better if calling speex_resampler_skip_zeros() after speex_resampler_init() (instead of on sample rate change - Bug 886653) and also getting the last latency samples from the resampler. Currently, it looks like the resampling code pads the stream with zeros at the beginning and perhaps truncates a little short of the end. Things are not going to be completely seamless unless we merge streams of the same sample rate and doppler effect before the resampler, but it looks like we can do a lot better before resorting to that.
Yeah, you shouldn't necessarily trust that I've been using the speex resampler correctly at all. :-)
Anthony, can you compare the .sampleRate of an AudioContext in Firefox and Chrome on your machine? On my machine Chrome defaults to 44100 and Firefox defaults to 48000, so Chrome matches your buffer sample rate and Firefox doesn't. That may be the only difference here. It's rather awkward to try to find a way to automatically gather together AudioBufferSourceNodes based on their sample rates and resample them as a unit. That comes perilously close to supporting variable sample rates natively within an AudioContext. There's a thread about this on public-audio under "How to play back synthesized 22kHz audio in a glitch-free manner?" I think the best way to solve this use-case is to support creating an AudioContext with a particular sample rate. That should be pretty straightforward to implement in Gecko.
fwiw, I've got some code for cubeb somewhere that gives you the native samplerate of the audio backend. If needed, I can finish it.
(In reply to comment #5) > fwiw, I've got some code for cubeb somewhere that gives you the native > samplerate of the audio backend. If needed, I can finish it. That would only fix this test case. The same problem will happen if Anthony was using 48KHz buffers, right?
That would fix some cases without to do anything else, but yes, more work is needed.
It appears to me that there is a lot of static even when the frequencies match. Ex: http://toadking.com/audiocontext_test.html Both the AudioContext and the buffer frequencies are 48000, but there is still a lot of static on Firefox Nightly.
Those Emscripten-ized testcases are difficult to figure out. A plain HTML+JS testcase would be very useful.
Attached file simple testcase (obsolete) (deleted) —
This tescase produces what sounds like a continuous tone to me. How about you?
Flags: needinfo?(inolen)
Attached file testcase (deleted) —
it appears you uploaded some unrelated patch anyway, I got a testcase working with just javascript. It seems to be not just a WebAudio that's the issue, but a combination of requestAnimationFrame and AudioContext.currentTime. Adding a delay can fix it for Firefox, but Chrome does not need this.
Attachment #782044 - Attachment mime type: text/plain → text/html
Your testcase relies on requestAnimationFrame firing at 60fps and also relies on the audio clock (AudioContext.currentTime) advancing at the same rate as the clock used by requestAnimationFrame. Neither of those things is guaranteed.
Attached file testcase (deleted) —
Sorry about attaching the wrong file before. Here's my testcase. You can try reducing bufferDuration to 16ms to make this testcase behave more like your testcase. You could also reduce safetyBuffering quite a lot although it'll still be needed since main-thread delays can happen --- depending on the page and what else the browser is doing.
Attachment #781465 - Attachment is obsolete: true
Whiteboard: [blocking-webaudio-]
While testing for the pre-beta sign-off of this feature, I tried to verify this bug, so here are my questions and results, with latest Aurora (build ID: 20130902004002) on a Mac OSX 10.9 in 32bit mode machine: - the first testcase sounds the same with Aurora and Chrome - the second testcase doesn't "do" anything with Chrome - both links from comment 0 close themselves (the tabs in which are loaded) at end in Chrome, but not with Aurora; also, with Aurora I can still hear some clicking / popping noises - the link from comment 8 makes some noise with Chrome, but not with Aurora Does anyone have any suggestions/thoughts? Thanks!
Flags: needinfo?
QA Contact: manuela.muntean
Depends on: 913854
Flags: needinfo? → needinfo?(ehsan)
Flags: needinfo?(ehsan)
In the mean time have there been alternate tests given out with resampling done in JS rather than by firefox ahead of time?
(In reply to comment #15) > In the mean time have there been alternate tests given out with resampling done > in JS rather than by firefox ahead of time? Not that I know of.
Priority: -- → P2
Status: UNCONFIRMED → NEW
Ever confirmed: true
I just tried both roc's testcase from comment 13 and the testcase from comment 11, and both sounded good to me FF43. If you're still seeing this, please reopen or file a new bug, and we'll look at it ASAP. Thanks
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WORKSFORME
Flags: needinfo?(inolen)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: