Address fingerprinting issues with AudioContext
Categories
(Core :: Web Audio, enhancement, P2)
Tracking
()
People
(Reporter: padenot, Assigned: karlt)
References
(Depends on 2 open bugs)
Details
(Keywords: perf-alert, Whiteboard: [tor 13017][fingerprinting][fp-triaged][fpp:m5])
Attachments
(13 files, 2 obsolete files)
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details |
Reporter | ||
Updated•8 years ago
|
Reporter | ||
Comment 1•8 years ago
|
||
Updated•7 years ago
|
Comment 3•7 years ago
|
||
Updated•6 years ago
|
Comment 4•6 years ago
|
||
Reporter | ||
Comment 5•6 years ago
|
||
Updated•6 years ago
|
Updated•6 years ago
|
Comment 7•6 years ago
|
||
See [1]. Tom, I emailed you a copy.
Safari running in iOS, by default, automatically suspend new AudioContext’s that are not created in response to a tap-in display. Since our fingerprinting code is simply executed, this explains the existence of null values
The tests are a little old, Safari results are for v10 and v11. Assuming Safari still do this, as a first defense against Audio FP'ing, could implement this approach?
[1] A Web Browser Fingerprinting Method Based on the Web Audio API
~ The British Computer Society 2019
~ JORDAN S. QUEIROZ and E DUARDO L. FEITOSA
~ doi:10.1093/comjnl/bxy146
This extension seems to be a temporary solution
https://addons.mozilla.org/en-US/firefox/addon/audioctx-fingerprint-defender/
I would instead look at the methods/code used in CanvasBlocker. I believe the above addon is "flawed" and not very effective (from my travels and reading) - instead look at [1], [2] for better insight
[1] https://addons.mozilla.org/firefox/addon/canvasblocker/
[2] https://github.com/kkapsner/CanvasBlocker/
Comment 8•5 years ago
|
||
Related tickets (if anyone wants to link them)
- Bug 1324545 - timing
- Bug 1595823 -
ac-outputLatency
Not sure if this was intentional (in tests on my windows machine)
- FF61 and lower:
an-smoothingTimeConstant
gave high precision: 15 decimal places - FF62+:
an-smoothingTimeConstant
is 1 decimal place
If an-smoothingTimeConstant
is reduced to 1 decimal place precision for all OSes, that's great (couldn't quickly spot a bugzilla)
Reporter | ||
Comment 9•5 years ago
|
||
(In reply to Simon Mainey from comment #8)
Not sure if this was intentional (in tests on my windows machine)
- FF61 and lower:
an-smoothingTimeConstant
gave high precision: 15 decimal places- FF62+:
an-smoothingTimeConstant
is 1 decimal place
I find this really surprising.
Comment 10•5 years ago
|
||
(In reply to Paul Adenot (:padenot) from comment #9)
I find this really surprising.
Just to clarify: these are all "essentially" vanilla profiles, so no RFP (minor tweaks e.g. stop auto-updates), Firefox portables, final versions per major release - including ESRs - from FF60 through to Nightly: I have the same on a Linux Mint VM. I'll check those and get back to you.
My android is identical to windows machine (except for the addition of outputLatency and ac-sampleRate)
https://ghacksuserjs.github.io/TorZillaPrint/TorZillaPrint.html#audio - [run tests], show details
ac-baseLatency: 0
ac-sampleRate: 48000
ac-state: suspended
ac-maxChannelCount: 2
ac-numberOfInputs: 1
ac-numberOfOutputs: 0
ac-channelCount: 2
ac-channelCountMode: explicit
ac-channelInterpretation: speakers
an-fftSize: 2048
an-frequencyBinCount: 1024
an-minDecibels: -100
an-maxDecibels: -30
an-smoothingTimeConstant: 0.8
an-numberOfInputs: 1
an-numberOfOutputs: 1
an-channelCount: 1
an-channelCountMode: max
an-channelInterpretation: speakers
It'd be nice to get a larger set of results
Comment 11•5 years ago
|
||
FF 71.0 on Linux (Manjaro): 0.8
Comment 12•5 years ago
|
||
(In reply to Simon Mainey from comment #10)
My android is identical to windows machine (except for the addition of outputLatency and ac-sampleRate)
sorry: I meant ac-baseLatency , not ac-SampleRate
Comment 13•3 years ago
|
||
Now that Bug 531915 is fixed, we can pursue using the new fdlibm functions for Web Audio to eliminate the fingerprint.
Comment 14•3 years ago
|
||
194 Firefox AudioContext key results, excluding ac-outputLatency
and ac-sampleRate
which are covered by RFP, from the data set of A Study of Feasibility and Diversity of Web Audio Fingerprints. The data set is very light on Android and Linux results
CONTROL: 183 results
"ac-baseLatency":0,
"ac-channelCount":2,
"ac-channelCountMode":"explicit",
"ac-channelInterpretation":"speakers",
"ac-maxChannelCount":2,
"ac-numberOfInputs":1,
"ac-numberOfOutputs":0,
"ac-state":"suspended",
"an-channelCount":2,
"an-channelCountMode":"max",
"an-channelInterpretation":"speakers",
"an-fftSize":2048,
"an-frequencyBinCount":1024,
"an-maxDecibels":-30,
"an-minDecibels":-100,
"an-numberOfInputs":1,
"an-numberOfOutputs":1,
"an-smoothingTimeConstant":0.8,
DIFF: 7 results: all windows
"an-channelCount":1,
DIFF: 1 result: win10 x64; 78
"ac-maxChannelCount":0,
"an-channelCount":1,
DIFF: 1 result: win10 x64; 87
"ac-maxChannelCount":8,
1 result: win10 x64; 87
"ac-maxChannelCount":0,
1 result: win6.1 x64; 86
"ac-maxChannelCount":1,
That seems like a fairly thin short tail. Paul, can those two items be covered by RFP?
Reporter | ||
Comment 15•3 years ago
|
||
(In reply to Simon Mainey from comment #14)
DIFF: 7 results: all windows
"an-channelCount":1,
Thisi is real surprising. Per spec this defaults to 2
, so this is likely something the fingerprinting code sets.
DIFF: 1 result: win10 x64; 78
"ac-maxChannelCount":0,
"an-channelCount":1,
This is when running on a machine that has no audio hardware I think. I don't understand the an-channelCount
for the reasons described above.
DIFF: 1 result: win10 x64; 87
"ac-maxChannelCount":8,
This iis running on a machine that has a 7.1 soundcard.
1 result: win10 x64; 87
"ac-maxChannelCount":0,
Same as above.
1 result: win6.1 x64; 86
"ac-maxChannelCount":1
Again running with special hardware (mono audio device).
This ac-maxChannelCount
is already covered by RFP: https://searchfox.org/mozilla-central/source/dom/media/webaudio/AudioContext.cpp#714-716 (hardcode to stereo which is the overwhelming majority)
Comment 16•3 years ago
|
||
(In reply to Paul Adenot (:padenot) from comment #15)
Thisi is real surprising. Per spec this defaults to
2
, so this is likely something the fingerprinting code sets.
None of the samples would have been running RFP, especially given that none of the RFP per-OS results for ac-outputLatency
were present. So it's something else. Perhaps we can hardcode this to 2
if RFP is enabled to tighten things up, or debug why it doesn't return 2
without RFP. Who knows, maybe they were running some randomizing extension, which seems unlikely given other analysis
This
ac-maxChannelCount
is already covered by RFP: https://searchfox.org/mozilla-central/source/dom/media/webaudio/AudioContext.cpp#714-716 (hardcode to stereo which is the overwhelming majority)
Excellent. My bad.
Reporter | ||
Comment 17•3 years ago
|
||
(In reply to Simon Mainey from comment #16)
None of the samples would have been running RFP, especially given that none of the RFP per-OS results for
ac-outputLatency
were present. So it's something else. Perhaps we can hardcode this to2
if RFP is enabled to tighten things up, or debug why it doesn't return2
without RFP. Who knows, maybe they were running some randomizing extension, which seems unlikely given other analysis
It's always 2 unless set explicitely regardless of RFP, it's a normative spec requirement. It would be best to check the code to see where it sets it.
Comment 18•3 years ago
|
||
In this try run we might have figured out the AudioContext fingerprint; but https://audiofingerprint.openwpm.com/ still shows different OscillatorNode values for us. Our patches are here part one and part two.
I'm not sure what could be causing the difference; looking at OscillatorNode.cpp the only thing that jumps out to me in that file is float(2 * M_PI)
.... Paul do you have any ideas?
Comment 19•3 years ago
|
||
RFP alone currently alters these OscillatorNode and hybrid OscillatorNode/DynamicsCompressor (I assume because some AudioContext keys are hardcoded and used downstream). Also the WPM test fails to get outputLatency correctly and always reports it as 0 (due to order/lag in requesting the info) but I don't think this matters in these tests
Reporter | ||
Comment 20•3 years ago
|
||
Tom, this is probably caused by this particular using AnalyzerNode.getFloatFrequencyData
, and Firefox using a different FFT implementation than Chrome. We already compile the one they use (for libwebrtc
), and it's faster than the one we use, so there's hope.
Updated•2 years ago
|
Comment 21•2 years ago
|
||
Depends on D166626
Comment 22•2 years ago
|
||
Depends on D166710
Comment 23•2 years ago
|
||
Depends on D166711
Updated•2 years ago
|
Assignee | ||
Comment 24•2 years ago
|
||
The FFT implementation may be causing different fingerprints due to its use of system math libraries for sin() and cos(). If the goal is to minimize the fingerprinting entropy between Firefox users, then I would expect that persuading the FFT implementation to use fdlibm is more important than the choice of library.
The MOZ_LIBAV_FFT implementation is third party and regularly updated, so interposing math headers as was done with GDK headers may be a sustainable way to switch to fdlibm.
(In reply to Tom Ritter [:tjr] from comment #18)
In this try run we might have figured out the AudioContext fingerprint; but https://audiofingerprint.openwpm.com/ still shows different OscillatorNode values for us.
Was this between two Firefox users on the same OS?
Assignee | ||
Updated•2 years ago
|
Assignee | ||
Updated•2 years ago
|
Comment 25•2 years ago
|
||
The FFT implementation may be causing different fingerprints
FYI: AFAICT bug 1519004 changed the sum on my android. Previously I was 35.74996031448245
, now I am a new unseen before value of 35.74995414912701
(you can use https://audiofingerprint.openwpm.com/). Whether than changes android entropy, who knows
all platforms, from research
let knownGood = [
// .7383...
35.73832903057337, 35.7383295930922, 35.73833039775491, 35.73833402246237,
// .7499
35.74996018782258, 35.74996031448245, 35.7499681673944,
35.74995414912701, // <- my android 113+: 1519004 ?
]
Reporter | ||
Comment 26•2 years ago
|
||
Thanks for the confirmation, Simon, this was somewhat expected. The reason for changing this was unrelated to fingerprinting, it was a performance improvement, and consolidation of code to use less different fft libraries.
We have plans to move all of this again soon to another fft, this time using the same code on all platforms / CPU architectures, and there are other plans to change trig functions so that they're similar on all OSes / architectures as well.
Assignee | ||
Comment 27•2 years ago
|
||
openmax_dl's FFT, used on arm32, doesn't use system sin()
and cos()
because it has its own hard-coded tables for fixed and floating point.
We don't build ffvpx with CONFIG_HARDCODED_TABLES so av_rdft_init()
is currently using system libraries to initialize its tables.
Updated•2 years ago
|
Updated•2 years ago
|
Assignee | ||
Comment 28•2 years ago
|
||
What we need to do:
- Make ffvpx use fdlibm for irrational math functions, by interposing headers. I would hope that the precise functions such as floor() and fabs() do not show differences in return values across system libraries, and so we may not need to use fdlibm for these.
- Finish D166710 or probably more durably use the same header interposition for dom/media/webaudio.
- Check that there are no significant performance regressions. Performance comparisons of fdlibm have showed varied results [1][2], but, skimming through the use of irrational functions in Web Audio, I didn't see any that I expect to be critical for performance. Exponentials and powers for AudioParams might be the more sensitive operations. It would be nice if we don't need to put fdlibm behind a pref in Web Audio.
- Verify that these changes reduce the fingerprint to no more than already exists across different architectures. I'll likely need some help with this.
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=531915#c31
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=933257#c139
Updated•2 years ago
|
Assignee | ||
Updated•1 year ago
|
Assignee | ||
Comment 29•1 year ago
|
||
(In reply to Paul Adenot (:padenot) from comment #15)
(In reply to Simon Mainey from comment #14)
DIFF: 7 results: all windows
"an-channelCount":1,Thisi is real surprising. Per spec this defaults to
2
,
This was resolved in bug 1666490.
Comment 30•1 year ago
|
||
This was resolved in bug 1666490.
Thanks. That explains it. I just looked back at the dataset (which the researchers gave me). The paper was July 2021 and all those with "an-channelCount":1,
are FF78 or FF81 (guessing the tests took place around August 2020). Bug fix was FF83. Seems legit :)
Assignee | ||
Comment 31•1 year ago
|
||
This will facilitate making the functions available to C code.
Assignee | ||
Comment 32•1 year ago
|
||
Depends on D185777
Assignee | ||
Comment 33•1 year ago
|
||
These operations are used from dom/media/webaudio.
Depends on D185778
Assignee | ||
Comment 34•1 year ago
|
||
This provides a way to use fdlibm from third party C code without
modification, such as media/ffvpx, which is regularly updated and so this
avoids patching the vendored library.
A similar approach for C++ gets more complicated and is in a subsequent patch.
Depends on D185779
Assignee | ||
Comment 35•1 year ago
|
||
This includes the rdft FFT implementation.
Depends on D185780
Assignee | ||
Comment 36•1 year ago
|
||
This will facilitate switching these functions to fdlibm, which maps to math.h.
Depends on D185781
Assignee | ||
Comment 37•1 year ago
|
||
Depends on D185782
Assignee | ||
Comment 38•1 year ago
|
||
Depends on D185783
Assignee | ||
Comment 39•1 year ago
|
||
to avoid risk of log(2^order) / M_LN2 < order.
Depends on D185782
Assignee | ||
Comment 40•1 year ago
|
||
Depends on D185892
Assignee | ||
Comment 41•1 year ago
|
||
I was seeing hypot undefined in object files so NS_hypot's __builtin_hypot
call does not appear to be inlining the implementation.
Depends on D185893
Assignee | ||
Comment 42•1 year ago
|
||
Depends on D185894
Assignee | ||
Updated•1 year ago
|
Comment 43•1 year ago
|
||
Comment 44•1 year ago
|
||
Updated•1 year ago
|
Updated•1 year ago
|
Comment 45•1 year ago
|
||
Comment 46•1 year ago
|
||
Comment 47•1 year ago
|
||
bugherder |
Comment 48•1 year ago
|
||
Comment 49•1 year ago
|
||
Comment 50•1 year ago
|
||
Assignee | ||
Updated•1 year ago
|
Comment 51•1 year ago
|
||
Comment 52•1 year ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/cd6b30e9acce
https://hg.mozilla.org/mozilla-central/rev/60768557aa15
https://hg.mozilla.org/mozilla-central/rev/969e577218dc
https://hg.mozilla.org/mozilla-central/rev/b6a176bcaf0b
https://hg.mozilla.org/mozilla-central/rev/6eec009a00bf
https://hg.mozilla.org/mozilla-central/rev/05da0ca6db46
Comment 53•1 year ago
|
||
:karlt do you think this could be nominated for a relnote? https://wiki.mozilla.org/Release_Management/Release_Notes_Nomination
Comment 54•1 year ago
|
||
(In reply to Pulsebot from comment #50)
Pushed by ktomlinson@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/b6a176bcaf0b
use fdlibm for inexact math functions in webaudio r=padenot
== Change summary for alert #39288 (as of Sat, 12 Aug 2023 11:10:34 GMT) ==
Improvements:
Ratio | Test | Platform | Options | Absolute values (old vs new) | Performance Profiles |
---|---|---|---|---|---|
11% | webaudio | linux1804-64-shippable-qr | fission webrender | 134.17 -> 118.83 | Before/After |
For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=39288
Hi Karl! Do you think the changes introduced by b6a176bcaf0bb5d9c9035ff8bed63bf0e79da30c could have caused this improvement ?
Updated•1 year ago
|
Assignee | ||
Comment 55•1 year ago
|
||
(In reply to Alex Finder from comment #54)
Hi Karl! Do you think the changes introduced by b6a176bcaf0bb5d9c9035ff8bed63bf0e79da30c could have caused this improvement ?
Thank you for pointing out that. Yes b6a176bcaf0bb5d9c9035ff8bed63bf0e79da30c would have been the cause of those changes.
For Linux and Mac, a comparison is available with a slightly narrower range.
Linux results have improved across all subtests. The most notable differences are:
Base | New | Delta | Confidence | Total Runs | |
---|---|---|---|---|---|
Geometric Mean | 135 ± 1.24 | 118.33 ± 0.87 | -12.35% | 20.76 (high) | 6 / 6 |
Granular synthesis | 300.67 ± 1.78 | 247.17 ± 1.82 | -17.79% | 18.75 (high) | 6 / 6 |
Periodic Wave with Automation | 293.67 ± 1.75 | 180 ± 0.86 | -38.71% | 52.00 (high) | 6 / 6 |
Simple mixing (100 different buffers) | 670.5 ± 1.75 | 628.33 ± 0.84 | -6.29% | 8.02 (high) | 6 / 6 |
Simple mixing (100x same buffer) | 665.67 ± 2.25 | 623.33 ± 1.37 | -6.36% | 6.02 (high) | 6 / 6 |
Stereo Panning with Automation | 245.17 ± 1.64 | 140 ± 0.45 | -42.90% | 63.29 (high) | 6 / 6 |
Substractive synth | 454.33 ± 1.61 | 266 ± 0.79 | -41.45% | 60.64 (high) | 6 / 6 |
Synth | 677.33 ± 1.96 | 545 ± 0.94 | -19.54% | 22.75 (high) | 6 / 6 |
For Mac, subtests are mixed, but there are a couple of large regressions. MacOS appears to have some different implementations of sin and cos and exp2f at least.
Base | New | Delta | Confidence | Total Runs | |
---|---|---|---|---|---|
Downmix without resampling (Stereo -> Mono) | 45.8 ± 1.83 | 57 ± 8.95 | 24.45% | 4.85 (med) | 5 / 5 |
Geometric Mean | 100.2 ± 0.83 | 102.6 ± 8.03 | 2.40% | 0.65 (low) | 5 / 5 |
Granular synthesis | 303.6 ± 0.5 | 282.4 ± 7.85 | -6.98% | 2.13 (low) | 5 / 5 |
Simple mixing (100 different buffers) | 719.4 ± 0.74 | 777 ± 7.04 | 8.01% | 2.34 (low) | 5 / 5 |
Simple mixing (100x same buffer) | 685.8 ± 0.69 | 739.6 ± 7.04 | 7.84% | 2.30 (low) | 5 / 5 |
Simple source test without resampling | 25.2 ± 3.32 | 23.4 ± 10.73 | -7.14% | 1.52 (low) | 5 / 5 |
Stereo Panning with Automation | 100.6 ± 1.33 | 130 ± 8.75 | 29.22% | 5.74 (high) | 5 / 5 |
WINNT results the comparison has a slightly larger range due to build failures on intermediate revisions, but I expect the changes are still due to the same revision.
I see only improvements (or unchanged) for 32-bit WINNT subtests:
Base | New | Delta | Confidence | Total Runs | |
---|---|---|---|---|---|
Geometric Mean | 118.4 ± 0.46 | 111.8 ± 0.75 | -5.57% | 14.76 (high) | 5 / 5 |
Granular synthesis | 338.4 ± 0.45 | 319 ± 0.63 | -5.73% | 17.28 (high) | 5 / 5 |
Simple mixing (100 different buffers) | 768.8 ± 0.43 | 750.2 ± 0.67 | -2.42% | 6.94 (high) | 5 / 5 |
Stereo Panning with Automation | 278 ± 0.57 | 120.6 ± 1.26 | -56.62% | 160.65 (high) | 5 / 5 |
Substractive synth | 354.6 ± 1.76 | 320.8 ± 0.26 | -9.53% | 12.03 (high) | 5 / 5 |
Synth | 673 ± 0.32 | 648.4 ± 0.48 | -3.66% | 14.55 (high) | 5 / 5 |
64-bit WINNT subtests have some notable regressions and a small improvement.
Base | New | Delta | Confidence | Total Runs | |
---|---|---|---|---|---|
Geometric Mean | 94.4 ± 2.76 | 95.8 ± 0.87 | 1.48% | 1.14 (low) | 5 / 5 |
Periodic Wave with Automation | 136.8 ± 0.61 | 134 ± 0 | -2.05% | 7.48 (high) | 5 / 5 |
Simple source test without resampling (Stereo and positional) | 39 ± 0 | 40 ± 1.77 | 2.56% | 3.16 (med) | 5 / 5 |
Stereo Panning with Automation | 90.6 ± 6.46 | 102.6 ± 1.11 | 13.25% | 4.50 (med) | 5 / 5 |
Substractive synth | 221.6 ± 0.94 | 228.4 ± 0.66 | 3.07% | 5.92 (high) | 5 / 5 |
Synth | 467.2 ± 0.95 | 543 ± 1.07 | 16.22% | 23.24 (high) | 5 / 5 |
I'll file some bugs to look at countering the regressions.
Assignee | ||
Comment 56•1 year ago
|
||
Results from https://github.com/arkenfox/user.js/issues/1701 and QA confirm that variations in AudioContext fingerprinting scripts across operating systems with fixed sample rates have been removed, as intended. The operating system is exposed to fingerprinters via other means but this result does assure that variations in OS libraries (potentially between versions or configurations) are no longer exposed through AudioContext.
Fingerprints still differ between ARM and Intel machines. The variation between x86 and amd64 on Windows 11 has been removed, which is an unexpected bonus. Remaining hardware differences are still plausible so we should keep a look out for them.
So there are notable reductions in entropy here, but the AudioContext
sampleRate
, outputLatency
, and maxChannelCount
carry potentially more entropy unless preferences are configured appropriately. The reductions here are of limited value unless privacy.resistFingerprinting = true, or fingerprintingProtection is enabled together with privacy.fingerprintingProtection.overrides = "+AudioContext,+AudioSampleRate", to remove these other sources of variation.
RFPTarget::JSMathFdlibm
would also need to be set (as with privacy.resistFingerprinting = true, or with fingerprintingProtection and privacy.fingerprintingProtection.overrides = "+AudioContext,+AudioSampleRate,+JSMathFdlibm") to avoid exposure elsewhere of the same entropy removed here.
Some of these other sources of entropy might be removed by default in some browsing modes in future changes.
Release Note Request (optional, but appreciated)
[Affects Firefox for Android]:
Yes.
[Why is this notable]:
[Suggested wording]:
Web Audio in Firefox now uses the FDLIBM math library on all systems to
improve anonymity with resist fingerprinting mode.
[Links (documentation, blog post, etc)]:
https://support.mozilla.org/en-US/kb/firefox-protection-against-fingerprinting
Updated•1 year ago
|
Comment 57•1 year ago
|
||
Added a slightly reworded note to the Nightly release notes. Keeping the relnote? flag open to keep it on the radar for inclusion in our final release notes.
Assignee | ||
Updated•1 year ago
|
Description
•