Closed Bug 1566389 Opened 5 years ago Closed 4 years ago

decodeAudioData doesn't skip information frames in MP3 audio

Categories

(Core :: Web Audio, defect, P2)

68 Branch
defect

Tracking

()

VERIFIED FIXED
83 Branch
Tracking Status
firefox83 --- verified

People

(Reporter: chris.needham, Assigned: padenot)

References

(Blocks 1 open bug, Regressed 1 open bug)

Details

Attachments

(13 files, 2 obsolete files)

(deleted), image/png
Details
(deleted), image/png
Details
(deleted), image/png
Details
(deleted), image/png
Details
(deleted), image/png
Details
(deleted), text/html
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36

Steps to reproduce:

decodeAudioData produces different results in Firefox compared to Chrome and Safari for MP3 audio that contains information frames. There's a demo page at [1], which compares the output from decodeAudioData with a command line waveform visualisation tool that I have written [2].

The waveform is rendered in a canvas, using data from waveform-data.js, a library that downsamples the audio to produce a visual representation [3].

A blog post [4] goes into more detail on the information frames issue.

We have an audio visualisation library [5] where this issue causes a slight visible difference between the playhead position you see and the audio you hear.

[1] https://waveform.prototyping.bbc.co.uk/waveform-data-issue-63/
[2] https://github.com/bbc/audiowaveform
[3] https://github.com/bbc/waveform-data.js
[4] https://thebreakfastpost.com/2016/11/26/mp3-decoding-with-the-mad-library-weve-all-been-doing-it-wrong/
[5] https://github.com/bbc/peaks.js

Actual results:

In Firefox, the output from decodeAudioData agrees with Ex. 1, where the audiowaveform command line tool does not skip information frames.

Expected results:

I expect the output from decodeAudioData to be the same as Ex. 2 in the web page, i.e., the decoder should skip information frames.

I believe that the right call here would be to just triage this issue right away, without attempting to reproduce it: IMO it has enough information to be actionable. Please NI if there is need for QA to attempt to confirm this issue.

Component: Untriaged → Audio/Video: Playback
Product: Firefox → Core
Component: Audio/Video: Playback → Web Audio
Attached image Screenshot 2019-09-19 at 13.42.44.png (deleted) —

This is what I have on Firefox Nightly (71), on Mac, a bit between Ex. 1 and Ex. 2. On what platform are you testing this?

We're going to change mp3 decoders soon however, so this might well be fixed.

Assignee: nobody → padenot
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(chris.needham)
Attached image firefox-69-linux.png (deleted) —
Attached image chrome-76-linux.png (deleted) —

I'm testing on Firefox 69.0 on Ubuntu 16.04 (see firefox-69-linux.png), where as I believe the right output should be as in chrome-76-linux.png.

I should add, the fix I made to audiowaveform involves reading encoding delay and padding data from a Xing or Info header, and skipping samples: https://github.com/bbc/audiowaveform/commit/59bc5d6a73c8e5c60cc5d9cad38d22ab373f7027#diff-91c0d8b3c188ac213c9de60687f1907eR623

Attached image firefox-69.0.1-windows10.png (deleted) —

And here's the output from Firefox 69.0.1 on Windows 10.

Flags: needinfo?(chris.needham)

In bug 1582271, we're switching to use FFMpeg's mp3 decoder for all platform. When this is done (all the patches have been accepted), we'll have consistent result on all platform. At this stage, if the result is not correct, we'll fix it so that it skips the right number of samples.

Depends on: 1582271
Attached image firefox-71.0a1-2019-10-07-windows.png (deleted) —

I see that bug 1582271 is resolved. Here's what I get now in Firefox 71 nightly (2019-10-07) on Windows 7, so this still differs from audiowaveform and Chrome.

Note from the 1628072 dupe: this affects looping by introducing an audible gap between loops.
Other browsers can playback looping audio seamlessly.
It did not happen before Firefox 71 so should be treated as a regression.

I agree. It's time to do the right thing now, we have landed our mp3 decoder based on ffmpeg, so we can have a single code path for all platforms.

Severity: normal → S3
Attachment #9167316 - Attachment is obsolete: true
Attachment #9167316 - Attachment is obsolete: false
Attachment #9167316 - Attachment description: Bug 1566389 - Set mFirstFrame after the mp3 metadata r?padenot → Bug 1566389 - Parse lame encoder delay and padding when demuxing MP3s. r?bryce

This doesn't work, it's never != 0

Depends on D85585

Per https://thebreakfastpost.com/2016/11/26/mp3-decoding-with-the-mad-library-weve-all-been-doing-it-wrong/, http://all-day-breakfast.com/m/ticks.mp3 is a file that has a noise burst at the very beginning and very end of the file, when decoded properly with the right skips/padding.

Attached is a web page that draws the very beginning and very end of the decoded file.

Attached file decodeaudioData-vbr.html (deleted) —

Depends on D91772

Attachment #9167316 - Attachment is obsolete: true
Attachment #9173636 - Attachment is obsolete: true
Pushed by padenot@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/085b748cfbaf Allow removing elements at the end of an AlignedAudioBuffer. r=jya https://hg.mozilla.org/integration/autoland/rev/8390ea4677a1 Parse lame encoder delay and padding when demuxing MP3s. r=bryce,jya https://hg.mozilla.org/integration/autoland/rev/19a04bdb5344 Store the parsed encoder delay and padding when demuxing an mp3 file. r=jya,bryce https://hg.mozilla.org/integration/autoland/rev/498a7c9cbc5f Tag compressed mp3 frames as EOS when it's the case. r=jya https://hg.mozilla.org/integration/autoland/rev/8db398c91c5a Send encoder delay and padding info to the mp3 decoder. r=jya https://hg.mozilla.org/integration/autoland/rev/a3bedd49288d Trim the beginning and end of mp3 files, r=jya https://hg.mozilla.org/integration/autoland/rev/116644341a76 Test that various mp3 files have exactly the correct duration. r=jya
Flags: needinfo?(padenot)
Blocks: 1669166
Pushed by padenot@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/0fe64b836e24 Allow removing elements at the end of an AlignedAudioBuffer. r=jya https://hg.mozilla.org/integration/autoland/rev/ec4e0024ab8d Parse lame encoder delay and padding when demuxing MP3s. r=bryce,jya https://hg.mozilla.org/integration/autoland/rev/6d36dae2ca8f Store the parsed encoder delay and padding when demuxing an mp3 file. r=jya,bryce https://hg.mozilla.org/integration/autoland/rev/89378e240d00 Tag compressed mp3 frames as EOS when it's the case. r=jya https://hg.mozilla.org/integration/autoland/rev/820744dbc7d1 Send encoder delay and padding info to the mp3 decoder. r=jya https://hg.mozilla.org/integration/autoland/rev/dd68ca2ed329 Trim the beginning and end of mp3 files, r=jya https://hg.mozilla.org/integration/autoland/rev/748e0d854d86 Test that various mp3 files have exactly the correct duration. r=jya
Pushed by padenot@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/253cdc0024a1 Allow removing elements at the end of an AlignedAudioBuffer. r=jya https://hg.mozilla.org/integration/autoland/rev/7ac585ebb2c1 Parse lame encoder delay and padding when demuxing MP3s. r=bryce,jya https://hg.mozilla.org/integration/autoland/rev/6fdb8b5c6ae8 Store the parsed encoder delay and padding when demuxing an mp3 file. r=jya,bryce https://hg.mozilla.org/integration/autoland/rev/7a4200237fb1 Tag compressed mp3 frames as EOS when it's the case. r=jya https://hg.mozilla.org/integration/autoland/rev/6ad90ab742bf Send encoder delay and padding info to the mp3 decoder. r=jya https://hg.mozilla.org/integration/autoland/rev/bc144d4e01d0 Trim the beginning and end of mp3 files, r=jya https://hg.mozilla.org/integration/autoland/rev/ac99b200090e Test that various mp3 files have exactly the correct duration. r=jya
Flags: needinfo?(padenot)
Flags: qe-verify+

Reproduced the initial issue using old Nightly without this fix. verified that using Firefox 83.0b3 the output from decodeAudioData is the same as Ex. 2 in the web page across platforms (Windows 10 64bit, macOS 11 and Ubuntu 18.04).

Status: RESOLVED → VERIFIED
Flags: qe-verify+
Regressions: 1714448
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: