Closed Bug 1673285 Opened 4 years ago Closed 1 year ago

Port libSoundTouch to RLBox

Categories

(Core :: Audio/Video, enhancement)

Firefox 84
enhancement

Tracking

()

RESOLVED FIXED
119 Branch
Tracking Status
firefox119 --- fixed

People

(Reporter: e5johnso, Assigned: wrv)

References

(Blocks 1 open bug, Regressed 1 open bug)

Details

Attachments

(8 files, 4 obsolete files)

(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:82.0) Gecko/20100101 Firefox/82.0

Steps to reproduce:

This is the bug to track the process of porting libSoundTouch to RLBox. Currently it still uses the unsandboxed libSoundTouch, but now it uses the RLBox API so that we can easily switch between unsandboxed and sandboxed versions of the library.

Why did you file this as a security bug?

Group: firefox-core-security → media-core-security
Component: Untriaged → Audio/Video
Flags: needinfo?(e5johnso)
Product: Firefox → Core

I marked it as a security bug as it concerns the sandboxing (and therefore security) of a media library, although this is more of a security infrastructure issue than an actual vulnerability. If this is incorrect, it can be marked as a non-security bug.

Flags: needinfo?(e5johnso)
Assignee: nobody → e5johnso
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

I can't edit the security checkbox, but we don't need this to be a security bug. None of the other sandboxing bugs are.

Unhiding per comment #3 / comment #4.

Group: media-core-security

Depends on D94698

Attached file Bug 1673285 - Remove old file (obsolete) (deleted) —

Depends on D99634

Depends on D99635

Seems like all the new patches should be squashes into a single patch (the original phab).

Flags: needinfo?(e5johnso)
Attached file Bug 1673285 - Update rlbox validators (obsolete) (deleted) —

Depends on D94698

Attachment #9209304 - Attachment is obsolete: true
Attachment #9193119 - Attachment is obsolete: true
Attachment #9192956 - Attachment is obsolete: true
Flags: needinfo?(e5johnso)
Attachment #9192955 - Attachment is obsolete: true
Attachment #9183690 - Attachment description: Bug 1673285 - Port libSoundTouch to RLBox → WIP: Bug 1673285 - Part 1: Retrofit AudioStream to use RLBoxed soundtouch.

Depends on D94698

Depends on D109843

Assignee: e5johnso → deian
Attachment #9211679 - Attachment description: WIP: Bug 1673285 - Part 2: Add support for Wasm sandboxing soundtuch. → Bug 1673285 - Part 2: Add support for Wasm sandboxing soundtuch. r=tjr
Attachment #9211680 - Attachment description: WIP: Bug 1673285 - Part 3: Turn on Wasm sandboxing for soundtouch. → Bug 1673285 - Part 3: Turn on Wasm sandboxing for soundtouch. r=tjr
Attachment #9183690 - Attachment description: WIP: Bug 1673285 - Part 1: Retrofit AudioStream to use RLBoxed soundtouch. → Bug 1673285 - Part 1: Retrofit AudioStream to use RLBoxed soundtouch.
Attachment #9211679 - Attachment description: Bug 1673285 - Part 2: Add support for Wasm sandboxing soundtuch. r=tjr → Bug 1673285 - Part 2: Add support for Wasm sandboxing soundtuch.
Attachment #9211679 - Attachment description: Bug 1673285 - Part 2: Add support for Wasm sandboxing soundtuch. → Bug 1673285 - Part 3: Add support for Wasm sandboxing soundtuch.
Attachment #9211680 - Attachment description: Bug 1673285 - Part 3: Turn on Wasm sandboxing for soundtouch. r=tjr → Bug 1673285 - Part 4: Turn on Wasm sandboxing for soundtouch.

Depends on D109843

Attachment #9211680 - Attachment description: Bug 1673285 - Part 4: Turn on Wasm sandboxing for soundtouch. → Bug 1673285 - Part 5: Turn on Wasm sandboxing for soundtouch.
No longer blocks: Wasm-sandboxing
Attachment #9211679 - Attachment description: Bug 1673285 - Part 3: Add support for Wasm sandboxing soundtuch. → Bug 1673285 - Part 3: Add support for Wasm sandboxing soundtouch.

The bug assignee is inactive on Bugzilla, so the assignee is being reset.

Assignee: deian → nobody
Status: ASSIGNED → NEW

Combines previous Patches 1-4, adapted to the current codebase.

Assignee: nobody → wrv
Status: NEW → ASSIGNED

Part of patch 4 and patch 5 from previous effort.

Depends on D183627

This patch enables SIMD in Wasm sandboxed SoundTouch by using
simd-everywhere to map SSE intrinsics to WASMSIMD intrinsics and
compiling with -msimd128 flag.

Depends on D183628

These collection of patches apply the previous effort to the most recent Mozilla codebase, and also enable SIMD support for Wasm sandboxed libSoundTouch.

For

(continuing the previous message)

For completeness, I'm copying over https://bugzilla.mozilla.org/show_bug.cgi?id=1829765#c11 :


We have some SoundTouch numbers we can now share [1].

We calculated these numbers by following the methodology described in [2,3]:

  1. Open [4] and click 'set rate and play'. This will start playing a sound at 0.25x speed. Each time we click 'set rate and play' the speed will increase by 1x, up to 3.25x.
  2. Begin collecting Profiling information with Firefox Profiler
  3. Let the audio play completely at each speed, then click 'set rate and play' to go to the next speed.
  4. Once it stops playing, stop Profiling.
  5. Use [5] to plot Audio Callback values and get statistics.

Note that Native+SIMD is the default in Firefox.

Results:

 **Mean time (s)**
          No SIMD |   SIMD
        -------------------
 Native | 0.03382 | 0.01486 |
        |-------------------|
  Wasm  | 0.04419 | 0.01745 |
        -------------------
**Median time (s)**
          No SIMD |   SIMD
        -------------------
 Native | 0.03304 | 0.01386 |
        |-------------------|
  Wasm  | 0.04165 | 0.01473 |
        -------------------

The above Mean time numbers show that an RLBoxed SoundTouch with SIMD (0.01745s) gives us a 2.5x speedup compared to an RLBoxed SoundTouch without SIMD (0.04419s). Previous results compared native+SIMD (0.01486s) to RLBoxed SoundTouch without SIMD (0.04419s) giving an almost 200% overhead. Compared to native+SIMD (0.01486s), RLBoxed SoundTouch with SIMD (0.01745s) has a 17% overhead. The linked to Google docs [1] has some more details.

[1] New results: https://docs.google.com/document/d/19LtpeSWA6ghfAEGw1lN8im2QPMc7wL2W7LusXPzfg18/edit
[2] Methodology: https://blog.paul.cx/post/profiling-firefox-real-time-media-workloads/#the-new-solution
[3] Previous results: https://docs.google.com/document/d/1h4UsI_9bXrMS3jy1dtu83GLwdi7xB3bNYrWh2MdyEes/edit
[4] Benchmark website: https://ritter.vg/misc/ff/soundtouch-perf.html
[5] Audio callback plotting plugin: https://github.com/padenot/fx-profiler-audio-cb

Attachment #9343985 - Attachment description: Bug 1673285 - Enable Wasm sandboxing for SoundTtouch. r=padenot,shravanrn,glandium,deian → Bug 1673285 - Enable Wasm sandboxing for SoundTouch. r=padenot,shravanrn,glandium,deian
Pushed by mh@glandium.org: https://hg.mozilla.org/integration/autoland/rev/f66c9e9aad96 Retrofit AudioStream and AudioDecoderInputTrack to use RLBoxed SoundTouch. r=padenot,glandium https://hg.mozilla.org/integration/autoland/rev/f28e1fe0311f Enable Wasm sandboxing for SoundTouch. r=glandium https://hg.mozilla.org/integration/autoland/rev/6c2e3da74c76 Enable SIMD for Wasm sandboxed SoundTouch. r=glandium
Regressions: 1850427

Can you fix the problem with AudioStream.cpp and also update rlfox_wasm2c_sandbox to https://github.com/PLSysSec/rlbox_wasm2c_sandbox/commit/0938ae96c5467bdf2210cf21be47ae576cbcc3cd ?

Flags: needinfo?(mh+mozilla) → needinfo?(wrv)
Flags: needinfo?(wrv)
Pushed by mh@glandium.org: https://hg.mozilla.org/integration/autoland/rev/9cde99c7ec6a Retrofit AudioStream and AudioDecoderInputTrack to use RLBoxed SoundTouch. r=padenot,glandium https://hg.mozilla.org/integration/autoland/rev/ac7b4c876c2c Enable Wasm sandboxing for SoundTouch. r=glandium https://hg.mozilla.org/integration/autoland/rev/0e326fb37f8c Enable SIMD for Wasm sandboxed SoundTouch. r=glandium
Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → 119 Branch
Regressions: 1850640
Regressions: 1851107
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: