Grant Galitz Reporter
	Description • 13 years ago

User-Agent:       Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0) Gecko/20100101 Firefox/4.0
Build Identifier: 

A Windows NT specific bug it seems that causes audio output to misbehave on some Windows based machines tested. Strange gaps that seem to be caused by improper audio handling by the OS or Firefox itself.

Reproducible: Always

Steps to Reproduce:
1. Load the site inside Firefox 4 on Windows XP
2. The samples currently buffered should be in the thousands if the API was detected.

Actual Results:  
Gaps in the audio stream even though the metrics done in JavaScript show it should be continuous.

Expected Results:  
A smooth continuous stream of synthesized audio.

Works perfectly on Firefox 4 for Mac OS X.

Also works well on web audio builds of webkit for Mac OS X.

	Mardeg
	Comment 1 • 13 years ago

Mozilla/5.0 (Windows NT 5.1; rv:2.0) Gecko/20100101 Firefox/4.0
Confirming

	Matthew Gregan [:kinetik]
	Comment 2 • 13 years ago

Thanks for the testcase!

I hear dropouts on all platforms.  The minimum buffer argument to XAudioServer needs to be change to at least 100000 for the test to work at all on my Linux system.

	Grant Galitz Reporter
	Comment 3 • 13 years ago

I don't hear dropouts on my mac, even when I lower it to 10000. Flash in other tabs does cause droppage as well as other system tasks outside of firefox itself (FYI: If you move your mouse around your doc on OS X you might hear crackling, that seems to be because of OS X taking up the system timer and affects other applications too).

	Grant Galitz Reporter
	Comment 4 • 13 years ago

I guess linux audio support is buggier than that of OS X, but better than Windows NT-based OSes I presume? :P

	Grant Galitz Reporter
	Comment 5 • 13 years ago

kinetik: Interesting fact, the sample rate provided is 100khz, so maybe a bug in firefox 4 for linux?

	Matthew Gregan [:kinetik]
	Comment 6 • 13 years ago

One source of problems is in the test itself, which is dropping samples when mozWriteAudio's buffers are full:

XAudioServer#executeCallback calculates the required number of samples as:
    minBufferSize - (samplesWritten - mozCurrentSampleOffset)

underRunCallback generates the requested number of samples and returns the sample buffer, which is passed directly to mozWriteAudio:

    samplesWritten += mozWriteAudio(this.underRunCallback(samplesRequested));

It's possible for mozWriteAudio to return less than samplesRequested here, resulting in the remaining samples returned by underRunCallback being dropped.

Since the code attempts to have minBufferSize samples pending at any one time, the chance of this happening depends on the sizing of the audio backend's buffers.

On OS X, it's possible to write 882000 samples before a short write will occur.  On Win32, that value is 200000 samples.  On Linux, the value depends on the sample rate and format (and may depend on the audio hardware), but will usually be 100000 samples for this testcase.  Given the configuration of the testcase, it's likely that this bug in the test will only occur on Linux.

	Grant Galitz Reporter
	Comment 7 • 13 years ago

But I have it only refill when it hits 25k samples, that should be ok leg-room, no? I designed it on purpose to bounce samples silently if the audio subsystem taps out. The audio gaps though are occurring where if samples were being dropped you'd here skipping rather than dropping (I keep track of the samples only accepted by the API).

	Grant Galitz Reporter
	Comment 8 • 13 years ago

So why the dropping rather than a skipping sound?

	Grant Galitz Reporter
	Comment 9 • 13 years ago

*So why the gaps of silence rather than a skipping-like sound?

	Grant Galitz Reporter
	Comment 10 • 13 years ago

Anyhow, is there an exposed property for the firefox-internal buffer size that it sizes differently on each platform?

	David Humphrey (:humph)
	Comment 11 • 13 years ago

(In reply to comment #10)
> Anyhow, is there an exposed property for the firefox-internal buffer size that
> it sizes differently on each platform?

No.  The backend stuff is currently being rewritten, and I can't speak to its design.  Matthew knows.  I think that finding some solution to the "audio buffering is so different per platform" problem is worth exploring elsewhere.

	Matthew Gregan [:kinetik]
	Comment 12 • 13 years ago

(In reply to comment #7)
> But I have it only refill when it hits 25k samples, that should be ok leg-room,
> no?

I only see code that writes |minBufferSize - remainingBuffer| every time executeCallback is called, which is driven by a 16ms interval timer in the testcase.

The reliable way to avoid this problem is to expose the buffer size via the API, since computing it from the playback position and frames written is not reliable because the playback position can be before the start of the buffer.  We have this information internally, so it would just need to be exposed as a moz-prefixed attribute.

The next issue, and the major one in this case, is that the buffering model for the Win32 and Linux backends differs from the OS X backend.  The OS X backend acts as if there is a single buffer, and writing any amount of audio begins playback.  If there is not sufficient audio available and an underrun occurs, silence is inserted.

Win32 and Linux act as if there are multiple buffers (and the buffer size varies per platform and, on Linux, by OS audio implementation and potentially audio hardware).  A buffer must be filled before it is queued for playback.  This means that there is a minimum amount that must be written before playback begins, and to recover from an underrun.  Since this isn't exposed via the API, it's difficult to reliably recover from underruns right now.

With this particular testcase, 125ms of audio is buffered initially, then ~16ms extra is written every ~16ms.  This is enough to fill 1.25 buffers on Win32, so playback works but, assuming perfect 16ms intervals, there can be as little as 28ms of audio buffered and playable before an underrun occurs.  On Linux, the testcase doesn't work at all without modification because the initial start threshold is 500ms.

Adding logging into the underrun codepath in the Win32 backend confirms that the testcase is underrunning quite often in a 32-bit Vista VMWare VM running on an idle 2.66GHz MacBookPro.  Increasing the testcase's minimum buffer size to 40000 samples (2 complete internal buffers) is sufficient to avoid underruns most of the time on that machine.  Logging added to the testcase shows that generateAudio is only being called every ~24-30ms.

I think the best way forward is to expose the internal buffer size (or make it a fixed value and document it), and to modify the Win32 and Linux backends to behave the same way as the OS X backend.  The second item is part of what I'm working on in bug 623444.

	Grant Galitz Reporter
	Comment 13 • 13 years ago

And to show how the web audio api on the webkit side is performing: http://code.google.com/p/chromium/issues/detail?id=81445

Seems mac os x works well with the testcase, windows and linux give it nightmares for sure.

	Grant Galitz Reporter
	Comment 14 • 13 years ago

Workaround for windows firefox added by doing 2x prebuffer + minbuffer, rather than just minbuffer. This bug is still valid because we really shouldn't have to do this nasty prebuffer stuff for the audio logic to work right.

	Matthew Gregan [:kinetik]
	Comment 15 • 12 years ago

Can you please test a recently nightly (or one of the new Aurora 15 builds once they're out) and let me know if the situation has improved?

	Grant Galitz Reporter
	Comment 16 • 12 years ago

I was still experiencing this problem in the nightlies on Windows 7 and in Ubuntu. I've gone ahead and blacklisted the mozilla audio data api and forced any user not on mac os x to use the flash fallback instead: https://github.com/grantgalitz/XAudioJS/commit/51443da66b554a78fe67ccb072c37450221c1fb7

I was receiving various complaints on locked up firefox and/or gappy audio when audio was enabled, due to moz audio bugs. The affected systems basically are all platforms except mac os x desktop.

	Grant Galitz Reporter
	Comment 17 • 12 years ago

kinetik: If this indeed gets fixed, I will still keep the blacklisting around due to too many firefox versions having bugged out moz audio. Need a safe way to detect this.

	Matthew Gregan [:kinetik]
	Comment 18 • 12 years ago

When did you last test a nightly?  The changes that would have affected the behaviour of this have only landed recently.

	Grant Galitz Reporter
	Comment 19 • 12 years ago

Anyhow, that prebuffer is too large (500 ms is NOT okay) and should be treated as a bug. Moz Audio really should in my opinion abstract away any platform-specific buffering model and treat it as single-buffer (ring buffer), as detecting waveform position is critical to many low-latency audio applications.

	Grant Galitz Reporter
	Comment 20 • 12 years ago

(In reply to Matthew Gregan [:kinetik] from comment #18)
> When did you last test a nightly?  The changes that would have affected the
> behaviour of this have only landed recently.

The other day, did this just land? Comment 17 still applies.

	Matthew Gregan [:kinetik]
	Comment 21 • 12 years ago

Yes, major changes to the audio backends have landed over the past few weeks, including changes in the last couple of days.  It'd be helpful if you were specific about which nightly build you tested.

	Grant Galitz Reporter
	Comment 22 • 11 years ago

Firefox on WinXP still has audio bugs when we attempt to keep a stable audio buffer with mozWriteAudio at any buffer rate less than 100 ms in size.

	Matthew Gregan [:kinetik]
	Comment 23 • 11 years ago

The Audio Data API was deprecated in bug 855570, which shipped in Firefox 23.  Developers should now be targeting the standardized Web Audio API instead, and filing bugs against it as necessary.  There's no point keeping the Audio Data API bugs open as we'll never fix them and will eventually remove the code.  Closing on that basis.