The performance of AudioContext.decodeAudioData is very slow. Testing to decode an OGG music file that is 270KB (duration 53.7 seconds) in size took 85 msecs on an OSX Macbook laptop, and decoding the same file on the Flame Firefox OS reference device took 8.226 seconds. In comparison, using an <audio> element to decode is practically instantaneous on the same Firefox OS device. (but I can no longer that method, due to bug #654787) Since those two methods perform exactly the same work(?), it sounds like the existing <audio> method proves there would be an opportunity to optimize AudioContext.decodeAudioData() greatly?

I suspect the main difference between the audio element and the decodeAudioData API is that the audio element will start playing before decoding the whole file. Still the 100x difference between Macbook and Flame sounds very large. I wonder how much resampling is contributing here. (That would not take place if the source file rate matches the AudioContext rate.)

(In reply to Jukka Jylänki from comment #0) > Since those two methods perform exactly the same work(?), it sounds like the > existing <audio> method proves there would be an opportunity to optimize > AudioContext.decodeAudioData() greatly? As Karl said, they don't. Also, the resulting buffer will take 18MB of RAM, I think it's worth mentioning, AudioBuffers are stored in RAM. It would be great to have a profile, here, to check if Karl is right. It's pretty straightforward to profile on an FxOS device, but you need to enable arbitrary thread profiling (since Web Audio is off-main-thread, and decoding happens on workers).

How do I do such a profile? I have built my own B2G from trunk with MOZ_PROFILING=1, and to do a profile, I run ./profile.sh start -p b2g -t Compositor && sleep 5 && ./profile.sh start -p myAppName but that does not show up audio decoding activities. I tried to find info about arbitrary thread profiling at https://wiki.mozilla.org/FirefoxOS/Performance/Profiling and https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Profiling_with_the_Built-in_Profiler but nothing caught my eye.

Also tried with ./profile.sh start -p b2g && sleep 5 && ./profile.sh start -p myAppName but that didn't show anything related to Web Audio either.

./profile.sh start -p myAppName -t MediaStreamGrph -f stackwalk,js,leaf,threads

I am unable to get that to work. When I do a run with /profile.sh start -p myAppName -t MediaStreamGrph -f stackwalk,js,leaf,threads when I load up the resulting .sym file, I get the error Error in worker: Exception: TypeError: threads[0] is undefined (https://people.mozilla.org/~bgirard/cleopatra/js/parserWorker.js:933) I also tried with -t MediaStreamGraph (as a guess if missing 'a' was a typo), but that had the same result.

Is there a way to list the valid values for the -t parameter?

I now see that the error is caused at least when the given thread did not exist. Reported bug #1013324 to ask to improve the error message.

It's not a typo: http://dxr.mozilla.org/mozilla-central/source/content/media/MediaStreamGraph.cpp#1639 Threads in Gecko can't have more that 16 characters in their name for portability reasons.

Questions: 1. Is this is a regression? 2. What's the user impact to the partner app if we don't fix this? 3. What's the risk in fixing this?

(In reply to Jason Smith [:jsmith] from comment #10) > Questions: > > 1. Is this is a regression? No. > 2. What's the user impact to the partner app if we don't fix this? I'll let Jukka weight in. > 3. What's the risk in fixing this? Not sure, we have to agree what it is we have to fix and if we want to fix it. For that, first, we need a profile.

BenWa: see comments 3-7 above. can you help Jukka with profiling the MediaStreamGraph thread?

You need to use the names that are registered to the profiler: mxr.mozilla.org/mozilla-central/source/content/media/MediaStreamGraph.cpp#1511 So use MediaStreamGraph. Make sure you built with ac_add_options --enable-profiling. Then use -t MediaStreamGraph and that -f includes 'threads'.

Triage determines that we will wait to hear from Martin.

This issue prevents us from having seamlessly looping background music. We are determining blocker status for 1.4. If not for 1.4, we def want this for 2.0. Jukka will follow up regarding profiling.

Splitting the file up into chunks would mean you could start playing before decoding the whole thing. I also recommend profiling different codecs as you may see significant differences in performance. One could speculate that hardware codecs will be faster but that remains to be proven.

Looks like both AAC and MP3 are software decoders as well.

Anthony - Can you find out what the partner app (game) impact here? I can't pinpoint how bad this is.

(In reply to Jason Smith [:jsmith] from comment #18) > Anthony - Can you find out what the partner app (game) impact here? I can't > pinpoint how bad this is. I believe the impact here is that there's a slight pause when an audio track loops in the app. My best guess is that the pause is about a half a second in duration.

Per Martin, there's a workaround, so we don't need to block here.

What is the workaround?

The workaround we had was a somewhat clumsy one: we used decodeAudioData only for short looping audio clips, and for everything else kept using the <audio> tag. This minimized the audible problems of bug #654787, although it was still present in long looping audio clips like background music.

This is not something we can fix. There is a need for a very flexible audio decoding API at a spec level, but this is not in scope for Web Audio API v1 (i.e., this is not going to happen anytime soon). A number of workarounds can be (and have been) used, so I'm going to close this.

Several Emscripten projects use an Emscripten-compiled audio decoder such as Ogg Vorbis, so those are not affected here since they can software decode in custom sized streams flexibly. I remember discussions about having a JS API for explicitly controlling decoding of subparts of an audio file (chunked decoding). That would mitigate the issue for those projects that don't use a asm.js-compiled decoder. What is the most recent discussion on being able to decode subparts of an audio file? Is that still a thing?

Jukka, the discussion is here [0], and we decided that it will be focused on after the spec stabilizes. The intent will be to have a lower-level, more flexible API. [0]: https://github.com/WebAudio/web-audio-api/issues/337