We need to add some unit tests to ensure that we are using the correct domains for search engines in all region/locales.

This should extend the work done in bug 1541415.

Depends on D30398

Backed out frequent xpcshell failures e.g. test_duckduckgo.js

https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=windows%2C7%2Cshippable%2Copt%2Cxpcshell%2Ctests%2Ctest-windows7-32-shippable%2Fopt-xpcshell-e10s-2%2Cx%28x2%29&tochange=73254a69497b209b372ed209ce9d004080ea8052&fromchange=e661822f67309406c695209d4be605ff281a2777&group_state=expanded&selectedJob=245750510 - the failure rate is somewhere at <=50% per number of retriggers, and they're either memory errors or thread creation errors after tests related to a search engine.
Example of each:

• https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=245731070&repo=autoland&lineNumber=6527
• https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=245750511&repo=autoland&lineNumber=247659

backout: https://hg.mozilla.org/integration/autoland/rev/44db460adb2b732a8444e0f35c968bd828313e12

push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=windows%2C7%2Cshippable%2Copt%2Cxpcshell%2Ctests%2Ctest-windows7-32-shippable%2Fopt-xpcshell-e10s-2%2Cx%28x2%29&tochange=73254a69497b209b372ed209ce9d004080ea8052&fromchange=e661822f67309406c695209d4be605ff281a2777&group_state=expanded&selectedJob=245746598

e.g failure snippet:
14:14:34 INFO - TEST-PASS | toolkit/components/search/tests/xpcshell/searchconfigs/test_amazon.js | test_searchConfig_amazon - [test_searchConfig_amazon : 108] Should be available for region: "tk" locale: "hy-AM" - true == true
14:14:34 INFO - <<<<<<<
14:14:34 INFO - c:\users\task_1557492576\appdata\local\temp\xpc-plugins-5dvxji could not be cleaned up.
14:14:34 INFO - c:\users\task_1557492576\appdata\local\temp\xpc-plugins-9_ejbx could not be cleaned up.
14:14:34 INFO - Following exceptions were raised:
14:14:34 INFO - Traceback (most recent call last):
14:14:34 INFO - File "Z:\task_1557492576\build\tests\xpcshell\runxpcshelltests.py", line 185, in run
14:14:34 INFO - self.run_test()
14:14:34 INFO - File "Z:\task_1557492576\build\tests\xpcshell\runxpcshelltests.py", line 715, in run_test
14:14:34 INFO - process_output, _ = self.communicate(proc)
14:14:34 INFO - File "Z:\task_1557492576\build\tests\xpcshell\runxpcshelltests.py", line 249, in communicate
14:14:34 INFO - self.process_line(line)
14:14:34 INFO - File "Z:\task_1557492576\build\tests\xpcshell\runxpcshelltests.py", line 602, in process_line
14:14:34 INFO - self.report_message(line_object)
14:14:34 INFO - File "Z:\task_1557492576\build\tests\xpcshell\runxpcshelltests.py", line 572, in report_message
14:14:34 INFO - self.output_lines.append(message)
14:14:34 INFO - MemoryError
14:14:34 INFO - Traceback (most recent call last):
14:14:34 INFO - File "Z:\task_1557492576\build\tests\xpcshell\runxpcshelltests.py", line 1669, in <module>
14:14:34 INFO - main()
14:14:34 INFO - File "Z:\task_1557492576\build\tests\xpcshell\runxpcshelltests.py", line 1664, in main
14:14:34 INFO - if not xpcsh.runTests(options):
14:14:34 INFO - File "Z:\task_1557492576\build\tests\xpcshell\runxpcshelltests.py", line 1431, in runTests
14:14:34 INFO - mobileArgs, **kwargs)
14:14:34 INFO - File "Z:\task_1557492576\build\tests\xpcshell\runxpcshelltests.py", line 1627, in runTestList
14:14:34 INFO - raise exceptions[0]
14:14:34 INFO - MemoryError
WARNING | IO Completion Port failed to signal process shutdown
Parent process 5964 exited with children alive:
PIDS: 3560
Attempting to kill them, but no guarantee of success
14:17:41 ERROR - Return code: 1
14:17:41 ERROR - No tests run or test summary not found
14:17:41 INFO - TinderboxPrint: xpcshell-xpcshell<br/><em class="testfail">T-FAIL</em>

I've been looking into this, and it seems the fundamental issue is that the harness can't cope with the amount of logging we're throwing at it.

I'm therefore working on some patches to only output the logging if we're in test debug mode, or if there's actual failures. These seem to help a lot.

I'm currently pondering between splitting the tests to run in 4 chunks per test, however that isn't really showing any gains at the moment, so I might stay with the structure as it is already.

To follow up on my previous comment, it seems splitting the tests to run in 4 chunks does make it faster on Windows & Mac. It is also better than leaving them as one chunk, with some extra parallel running being handled pretty well on Windows & Mac.

This splits running of locales across 4 chunks, which can run in parallel better.

The chunks can be run individually with '--tag=searchconfig1' etc.

It also stops logging test output in the pass cases (unless we're in _testDebug=true mode). This makes less work on the python harness which was causing a bottleneck and slowing the tests down.

Depends on D30399

https://hg.mozilla.org/mozilla-central/rev/afbd9e47febe
https://hg.mozilla.org/mozilla-central/rev/26e0b9e416db
https://hg.mozilla.org/mozilla-central/rev/0e0c4b2166dc