Improve profile and preference handling
Categories
(Testing :: Raptor, task, P1)
Tracking
(firefox68 fixed)
Tracking | Status | |
---|---|---|
firefox68 | --- | fixed |
People
(Reporter: Bebe, Assigned: whimboo)
References
Details
Attachments
(9 files, 1 obsolete file)
(deleted),
text/plain
|
Details | |
(deleted),
patch
|
Details | Diff | Splinter Review | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details |
Raptor is not setting preferences if the app is geckoview
We can set these preferences when creating the profile
Add this capability to raptor
https://searchfox.org/mozilla-central/source/testing/raptor/raptor/raptor.py#210
Updated•6 years ago
|
Comment 1•6 years ago
|
||
Good find :bebe! Yes so turns out no Raptor tests running on geckoview currently use custom prefs, that's why this hasn't been spotted before now. I thought 'ugl' used prefs but they don't - only some 'wasm' variations on Raptor desktop do, i.e. [0].
No rush to fix this but if any tests coming online on geckoview need custom prefs in the Raptor test INI then this should be fixed for that.
Assignee | ||
Comment 2•5 years ago
|
||
This blocks my work for streaming tests. So taking this bug.
Assignee | ||
Comment 3•5 years ago
|
||
Note, that this problem is not only for custom prefs in an INI file! Also some preferences as specified in testing/profiles/common/user.js
which should be used for Raptor, aren't present when geckoview_example is running. The profile at /sdcard/raptor/profile
contains all the user preferences.
Assignee | ||
Comment 4•5 years ago
|
||
This is actually a diff between the user.js file as generated by Raptor, and the prefs.js file as used by geckoview_example app. As you can see a lot of user preferences are not getting saved to prefs.js, and as such might not be used.
Assignee | ||
Comment 5•5 years ago
|
||
The problem here is actually not in Raptor, or how we use mozprofile and such, but the GeckoRuntimeSettings are resetting preferences even user defined ones. See bug 1547717. As such there is nothing we can do on that end.
But nevertheless while working on this bug, and trying to investigate the problem I found a couple of bugs, and also improvements Raptor could be happy with. So I will address those here.
Assignee | ||
Comment 6•5 years ago
|
||
Assignee | ||
Comment 7•5 years ago
|
||
Assignee | ||
Comment 8•5 years ago
|
||
The last try showed some minor issues which are now fixed here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=103c20906f848b2c8fad42317155af8c1dc5d923
Assignee | ||
Comment 9•5 years ago
|
||
Depends on D29329
Assignee | ||
Comment 10•5 years ago
|
||
Depends on D29330
Assignee | ||
Comment 11•5 years ago
|
||
Depends on D29331
Assignee | ||
Comment 12•5 years ago
|
||
Depends on D29332
Assignee | ||
Comment 13•5 years ago
|
||
The last try showed some minor issues which are now fixed here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=103c20906f848b2c8fad42317155af8c1dc5d923
Assignee | ||
Comment 14•5 years ago
|
||
We cannot land this patch before bug 1547933 has been fixed.
Updated•5 years ago
|
Assignee | ||
Comment 15•5 years ago
|
||
(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #14)
We cannot land this patch before bug 1547933 has been fixed.
Well that bug is actually a regression from my patch on this bug. We have to mock the ADBDevice in the tests to not require the adb
command which isn't available on those workers.
Maybe I get to it next week.
Assignee | ||
Comment 16•5 years ago
|
||
Assignee | ||
Comment 17•5 years ago
|
||
Due to missing class abstractions for Gecko- and non-Gecko based
browsers it's currently sub-optimal to define when preferences
as defined by tests will be set.
Given that by default we run Gecko-based applications other
browsers should opt-out from setting test preferences.
Depends on D30922
Assignee | ||
Comment 18•5 years ago
|
||
I will revert parts of the patch, so that we no longer will initialize the ADBDevice immediately, and as such don't run into those failures for Raptor python-tests.
Assignee | ||
Comment 19•5 years ago
|
||
There is so much to fix in the Raptor code, which actually makes it hard for me to not blow-up this patch series even more. I hope that the upcoming remaining patches are a good compromise for all of us. More improvements can always be done at a later point.
Assignee | ||
Updated•5 years ago
|
Assignee | ||
Comment 20•5 years ago
|
||
Depends on D29330
Assignee | ||
Comment 21•5 years ago
|
||
Last try build had a problem on Android, because I missed a return statement:
With the fix in place here a new try only for Android:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=1023140c10568fce42f32b195af996b238fcf5b4
Comment 22•5 years ago
|
||
Comment 23•5 years ago
|
||
Backed out changeset 90a51dbd6ee1 (Bug 1528236) for causing permafailures in raptor tests CLOSED TREE
Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=246473621&repo=autoland&lineNumber=1293
21:11:51 INFO - raptor-main Exception: TypeError launch_application() got an unexpected keyword argument 'moz_env'
21:11:51 INFO - raptor-main removing reverse socket connections
21:11:51 INFO - adb command_output: adb -s ZY3222K8ND wait-for-device reverse --remove-all, timeout: None, timedout: None, exitcode: 0, output:
21:11:51 INFO - raptor-main No crash directory (/sdcard/raptor/profile/minidumps) found on remote device
21:11:51 INFO - mozproxy Stopping mitmproxy playback, killing process 676
21:11:51 INFO - mozproxy Successfully killed the mitmproxy playback process
21:11:51 INFO - raptor-main removing webext /builds/task_1557868152/workspace/build/tests/raptor/raptor/../webext/raptor
21:11:51 INFO - raptor-main removing test folder for raptor: /sdcard/raptor
21:11:51 INFO - adb shell_output: adb -s ZY3222K8ND wait-for-device shell rm -r /sdcard/raptor; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output:
21:11:51 INFO - raptor-control-server shutting down control server
21:11:52 INFO - raptor-main finished
21:11:52 ERROR - Traceback (most recent call last):
21:11:52 INFO - File "/builds/task_1557868152/workspace/build/tests/raptor/raptor/raptor.py", line 1225, in <module>
21:11:52 INFO - main()
21:11:52 INFO - File "/builds/task_1557868152/workspace/build/tests/raptor/raptor/raptor.py", line 1193, in main
21:11:52 INFO - success = raptor.run_tests(raptor_test_list, raptor_test_names)
21:11:52 INFO - File "/builds/task_1557868152/workspace/build/tests/raptor/raptor/raptor.py", line 955, in run_tests
21:11:52 INFO - return super(RaptorAndroid, self).run_tests(tests, test_names)
21:11:52 INFO - File "/builds/task_1557868152/workspace/build/tests/raptor/raptor/raptor.py", line 186, in run_tests
21:11:52 INFO - self.run_test(test, timeout=int(test['page_timeout']))
21:11:52 INFO - File "/builds/task_1557868152/workspace/build/tests/raptor/raptor/raptor.py", line 974, in run_test
21:11:52 INFO - self.run_test_cold(test, timeout)
21:11:52 INFO - File "/builds/task_1557868152/workspace/build/tests/raptor/raptor/raptor.py", line 1062, in run_test_cold
21:11:52 INFO - self.launch_firefox_android_app(test['name'])
21:11:52 INFO - File "/builds/task_1557868152/workspace/build/tests/raptor/raptor/raptor.py", line 923, in launch_firefox_android_app
21:11:52 INFO - fail_if_running=False)
21:11:52 ERROR - TypeError: launch_application() got an unexpected keyword argument 'moz_env'
21:11:52 ERROR - Return code: 1
21:11:52 WARNING - setting return code to 1
21:11:52 INFO - Killing logcat pid 466.
21:11:52 CRITICAL - PERFHERDER_DATA was seen 0 times, expected 1.
21:11:52 INFO - copying raptor results to upload dir:
21:11:52 INFO - /builds/task_1557868152/workspace/build/blobber_upload_dir/perfherder-data.json
21:11:52 INFO - copying raptor results from /builds/task_1557868152/workspace/build/raptor.json to /builds/task_1557868152/workspace/build/blobber_upload_dir/perfherder-data.json
21:11:52 CRITICAL - Error copying results /builds/task_1557868152/workspace/build/raptor.json to upload dir /builds/task_1557868152/workspace/build/blobber_upload_dir/perfherder-data.json
21:11:52 INFO - [Errno 2] No such file or directory: u'/builds/task_1557868152/workspace/build/raptor.json'
21:11:52 INFO - Running post-action listener: _package_coverage_data
21:11:52 INFO - Running post-action listener: _resource_record_post_action
21:11:52 INFO - Running post-action listener: process_java_coverage_data
21:11:52 INFO - Running post-action listener: stop_device
21:12:01 INFO - /data/tombstones/dsps deleted
21:12:03 INFO - /data/tombstones/lpass deleted
Comment 24•5 years ago
|
||
Assignee | ||
Comment 25•5 years ago
|
||
I will have a look at this when I'm back on Friday. Most likely will push this to a different bug if necessary at all.
Comment 26•5 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/c6cccc4ebfcc
https://hg.mozilla.org/mozilla-central/rev/9eb4d019ee37
https://hg.mozilla.org/mozilla-central/rev/e8d6d453337d
https://hg.mozilla.org/mozilla-central/rev/09eea8c87cd5
https://hg.mozilla.org/mozilla-central/rev/ed679c1e266d
https://hg.mozilla.org/mozilla-central/rev/d12eb6d840fc
https://hg.mozilla.org/mozilla-central/rev/0d0a4250902b
Assignee | ||
Comment 27•5 years ago
|
||
(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #25)
I will have a look at this when I'm back on Friday. Most likely will push this to a different bug if necessary at all.
I filed bug 1552296 as a follow-up bug. So no further work is needed on this bug.
Updated•5 years ago
|
Description
•