Closed Bug 1716423 Opened 3 years ago Closed 3 years ago

[wpt-sync] Sync PR 29367 - Adjust test thresholds for Android

Categories

(Core :: Web Audio, task, P4)

task

Tracking

()

RESOLVED FIXED
91 Branch
Tracking Status
firefox91 --- fixed

People

(Reporter: mozilla.org, Unassigned)

References

()

Details

(Whiteboard: [wptsync downstream])

Sync web-platform-tests PR 29367 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/29367
Details from upstream follow.

b'Raymond Toy <rtoy@chromium.org>' wrote:

Adjust test thresholds for Android

Adjust the test thresholds for these tests so that they pass on
Android. The changes in the thresholds are deemed small enough not to
significantly degrade detection of actual errors. These seem like
small round-off errors in the implementation on Android that differs
from other platforms.

Bug: 1212851
Change-Id: I58e853d02285e8b46df85462336ab0463adeae5d
Reviewed-on: https://chromium-review.googlesource.com/2961534
WPT-Export-Revision: 3a3eaa6f5ca4abdd6e1d117a2f148243ff62d3a9

Component: web-platform-tests → Web Audio
Product: Testing → Core

CI Results

Ran 0 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 3 tests and 7 subtests

Status Summary

Firefox

OK : 3
PASS : 27
FAIL : 13

Chrome

OK : 3
PASS : 40

Safari

OK : 2
PASS : 35
TIMEOUT: 2
NOTRUN : 1

Links

GitHub PR Head
GitHub PR Base

Details

New Tests That Don't Pass

/webaudio/the-audio-api/the-biquadfilternode-interface/biquad-automation.html
X Output of bandpass filter with frequency automation does not equal [0,0.27222275733947754,0.1703483760356903,-0.4806261956691742,-0.2788437604904175,0.3957056999206543,0.10840324312448502,-0.39677220582962036,0.06871235370635986,0.41004475951194763,-0.1897629052400589,-0.3590146601200104,0.29248547554016113,0.2614542245864868,-0.376722127199173,-0.14195005595684052...] with an element-wise tolerance of {"absoluteThreshold":0.0000046455,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[1] 2.7207741141319275e-1 2.7222275733947754e-1 1.4534592628479004e-4 5.3392276129043557e-4 4.6454999999999998e-6
[2] 1.7092493176460266e-1 1.7034837603569031e-1 5.7655572891235352e-4 3.3845683905525301e-3 4.6454999999999998e-6
[3] -4.7884583473205566e-1 -4.8062619566917419e-1 1.7803609371185303e-3 3.7042528126036492e-3 4.6454999999999998e-6
[4] -2.8035214543342590e-1 -2.7884376049041748e-1 1.5083849430084229e-3 5.4094269147552214e-3 4.6454999999999998e-6
[5] 3.9142531156539917e-1 3.9570569992065430e-1 4.2803883552551270e-3 1.0817100577811786e-2 4.6454999999999998e-6
...and 2072 more errors.
Max AbsError of 2.1190106123685837e-1 at index of 1149.
[1149] -1.1468840390443802e-1 9.7212657332420349e-2 2.1190106123685837e-1 2.1797682220768748e+0 4.6454999999999998e-6
Max RelError of 9.6699462879811648e+1 at index of 1000.
[1000] -1.7538917064666748e-1 -1.7951907357200980e-3 1.7359397991094738e-1 9.6699462879811648e+1 4.6454999999999998e-6
: FAIL
< [automate-freq] 1 out of 1 assertions were failed.: FAIL
X Output of bandpass filter with Q automation does not equal [0,0.013114781118929386,0.047823384404182434,0.09765293449163437,0.15657125413417816,0.21899054944515228,0.27981746196746826,0.3345244824886322,0.379226952791214,0.4107542335987091,0.4267079830169678,0.42550167441368103,0.40637844800949097,0.3694050908088684,0.31544122099876404,0.24608469009399414...] with an element-wise tolerance of {"absoluteThreshold":0.0000010133,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[1] 1.4311081729829311e-2 1.3114781118929386e-2 1.1963006108999252e-3 9.1217733643547391e-2 1.0132999999999999e-6
[2] 5.3999401628971100e-2 4.7823384404182434e-2 6.1760172247886658e-3 1.2914220316554043e-1 1.0132999999999999e-6
[3] 1.1236446350812912e-1 9.7652934491634369e-2 1.4711529016494751e-2 1.5065117185756505e-1 1.0132999999999999e-6
[4] 1.8244987726211548e-1 1.5657125413417816e-1 2.5878623127937317e-2 1.6528336105527966e-1 1.0132999999999999e-6
[5] 2.5735962390899658e-1 2.1899054944515228e-1 3.8369074463844299e-2 1.7520881408379727e-1 1.0132999999999999e-6
...and 3994 more errors.
Max AbsError of 4.8764730989933014e-1 at index of 123.
[123] 5.8356797695159912e-1 9.5920667052268982e-2 4.8764730989933014e-1 5.0838607036959189e+0 1.0132999999999999e-6
Max RelError of 1.3733935694848860e+5 at index of 106.
[106] -4.0824136137962341e-1 -2.9724792511842679e-6 4.0823838890037223e-1 1.3733935694848860e+5 1.0132999999999999e-6
: FAIL
< [automate-q] 1 out of 1 assertions were failed.: FAIL
X Output of lowshelf filter with gain automation does not equal [0,0.4781356751918793,1.5511385202407837,3.0428154468536377,4.649440288543701,6.148099899291992,7.424892902374268,8.436473846435547,9.169538497924805,9.61856460571289,9.779438972473145,9.650873184204102,9.237476348876953,8.551753997802734,7.614615440368652,6.454819679260254...] with an element-wise tolerance of {"absoluteThreshold":0.000027657,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[1] 4.7863015532493591e-1 4.7813567519187927e-1 4.9448013305664063e-4 1.0341837238106155e-3 2.7657000000000000e-5
[2] 1.5550640821456909e+0 1.5511385202407837e+0 3.9255619049072266e-3 2.5307616655009381e-3 2.7657000000000000e-5
[3] 3.0560569763183594e+0 3.0428154468536377e+0 1.3241529464721680e-2 4.3517359813635154e-3 2.7657000000000000e-5
[4] 4.6794834136962891e+0 4.6494402885437012e+0 3.0043125152587891e-2 6.4616649076265492e-3 2.7657000000000000e-5
[5] 6.2022128105163574e+0 6.1480998992919922e+0 5.4112911224365234e-2 8.8015666808857178e-3 2.7657000000000000e-5
...and 2048 more errors.
Max AbsError of 2.4090213775634766e+0 at index of 120.
[120] 9.9098815917968750e+0 7.5008602142333984e+0 2.4090213775634766e+0 3.2116601413157825e-1 2.7657000000000000e-5
Max RelError of 1.0632292419525375e+2 at index of 1000.
[1000] 2.2968197241425514e-2 -2.1807405573781580e-4 2.3186271297163330e-2 1.0632292419525375e+2 2.7657000000000000e-5
: FAIL
< [automate-gain] 1 out of 1 assertions were failed.: FAIL
X Output of bandpass filter with detune automation does not equal [0,0.0008383856620639563,0.00141120795160532,0.00036609364906325936,0.0001000093761831522,0.0012290297308936715,0.001128630479797721,0.0000010075401633002912,0.0004436193557921797,0.0014331036945804954,0.00065815937705338,-0.00011537729005794972,0.0008997227996587753,0.0013498960761353374,0.00016446926747448742,0.00006718232179991901...] with an element-wise tolerance of {"absoluteThreshold":0.000031471,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[4] 1.3306002074386925e-4 1.0000937618315220e-4 3.3050644560717046e-5 3.3047545962280317e-1 3.1470999999999997e-5
[7] 6.9791625719517469e-5 1.0075401633002912e-6 6.8784085556217178e-5 6.8269323707065467e+1 3.1470999999999997e-5
[8] 4.8867485020309687e-4 4.4361935579217970e-4 4.5055494410917163e-5 1.0156340976254449e-1 3.1470999999999997e-5
[10] 7.0790149038657546e-4 6.5815937705338001e-4 4.9742113333195448e-5 7.5577610936569728e-2 3.1470999999999997e-5
[11] -4.4212661123310681e-6 -1.1537729005794972e-4 1.1095602394561865e-4 9.6167992756537768e-1 3.1470999999999997e-5
...and 1126 more errors.
Max AbsError of 1.0178004354238510e+0 at index of 1015.
[1015] -1.4570061862468719e-1 8.7209981679916382e-1 1.0178004354238510e+0 1.1670687412358907e+0 3.1470999999999997e-5
Max RelError of Infinity at index of 1087.
[1087] 1.0017648339271545e-1 0.0000000000000000e+0 1.0017648339271545e-1 Infinity 3.1470999999999997e-5
: FAIL
< [automate-detune] 1 out of 1 assertions were failed.: FAIL
X Output of peaking filter with automation of all parameters does not equal [0,0.9876883625984192,-0.30901700258255005,-0.8910065293312073,0.5877852439880371,0.7071067690849304,-0.80901700258255,-0.45399048924446106,0.9510565400123596,0.15643446147441864,-1,0.15643446147441864,0.9510565400123596,-0.45399048924446106,-0.80901700258255,0.7071067690849304...] with an element-wise tolerance of {"absoluteThreshold":0.00062907,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[697] -8.9100652933120728e-1 -8.9659482240676880e-1 5.5882930755615234e-3 6.2327965050708450e-3 6.2907000000000002e-4
[698] -3.0901700258255005e-1 -3.0482962727546692e-1 4.1873753070831299e-3 1.3736772716318364e-2 6.2907000000000002e-4
[699] 9.8768836259841919e-1 1.0072603225708008e+0 1.9571959972381592e-2 1.9430885475989618e-2 6.2907000000000002e-4
[700] -5.5879793522430055e-14 -3.7726949900388718e-2 3.7726949900332839e-2 9.9999999999851885e-1 6.2907000000000002e-4
[701] -9.8768836259841919e-1 -9.7246229648590088e-1 1.5226066112518311e-2 1.5657230277759220e-2 6.2907000000000002e-4
...and 3290 more errors.
Max AbsError of 1.0000780582427979e+1 at index of 2030.
[2030] 1.2776083946228027e+0 1.1278388977050781e+1 1.0000780582427979e+1 8.8672066575975750e-1 6.2907000000000002e-4
Max RelError of 7.2626681742283040e+1 at index of 898.
[898] -2.0910589396953583e-1 -2.8400830924510956e-3 2.0626581087708473e-1 7.2626681742283040e+1 6.2907000000000002e-4
: FAIL
< [automate-all] 1 out of 1 assertions were failed.: FAIL
X Output of bandpass filter with sinusoidal modulation of bandpass center frequency does not equal [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0...] with an element-wise tolerance of {"absoluteThreshold":0.000039787,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[1] 1.7610176000744104e-3 0.0000000000000000e+0 1.7610176000744104e-3 Infinity 3.9786999999999997e-5
[2] 6.9368327967822552e-3 0.0000000000000000e+0 6.9368327967822552e-3 Infinity 3.9786999999999997e-5
[3] 1.5246707014739513e-2 0.0000000000000000e+0 1.5246707014739513e-2 Infinity 3.9786999999999997e-5
[4] 2.6283653452992439e-2 0.0000000000000000e+0 2.6283653452992439e-2 Infinity 3.9786999999999997e-5
[5] 3.9528924971818924e-2 0.0000000000000000e+0 3.9528924971818924e-2 Infinity 3.9786999999999997e-5
...and 3993 more errors.
Max AbsError of 9.7427284717559814e-1 at index of 3596.
[3596] -9.7427284717559814e-1 0.0000000000000000e+0 9.7427284717559814e-1 Infinity 3.9786999999999997e-5
Max RelError of Infinity at index of 1.
: FAIL
< [modulation] 1 out of 1 assertions were failed.: FAIL

AUDIT TASK RUNNER FINISHED: 6 out of 6 tasks were failed.: FAIL

Pushed by wptsync@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/e5b0bcc5a07b [wpt PR 29367] - Adjust test thresholds for Android, a=testonly https://hg.mozilla.org/integration/autoland/rev/1f341d74f929 [wpt PR 29367] - Update wpt metadata, a=testonly
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → 91 Branch
You need to log in before you can comment on or make changes to this bug.