Like bug 1506523, Mochitest needs a few changes so that Thunderbird can use it. Mostly just using "mail:3pane" where "navigator:browser" is, and avoiding properties that don't exist in a Thunderbird window.

I've also put in the mozharness changes needed for a "thunderbird" suite of tests, so we can avoid running all the others.

Comment on attachment 9024581 [details] [diff] [review] 1506715-mochitest-1.diff Sorry, but I'm not the correct reviewer here. Geoff is a better choice here.

:darktrojan - Can you provide some context for me? Are there plans to run other test suites against thunderbird, or just this browser-chrome-thunderbird? Is browser-chrome-thunderbird going to run in continuous integration on mozilla-central, or ...? Why are we doing this now? (I don't know anything about Thunderbird development or testing!)

We plan to run only the browser-chrome-thunderbird tests on comm-central CI. We don't currently run any mochitests there. Thunderbird is not in mozilla-central, so it won't be affected.

OK, thanks. I think all the mochitest harness changes are fine: Make mochitest more flexible with respect to the application under test without harming the firefox support. (btw, I suspect we'll need to make similar changes to support geckoview test apps soon). But I don't understand the mozharness additions. It doesn't seem right to introduce thunderbird configuration alongside all the firefox configurations. Why is it needed? Does comm-central CI use linux_unittest.py, etc?

We do, and there are two Thunderbird-specific config files in that same directory. I admit mozharness really isn't my area, and I'm just hacking away until it works. Cc'ing Rob, because that's his thing.

Well, thunderbird_buildbot.py says that once Thunderbird is running on buildbot, that file can basically go away. I'll have to look into that. As to the *unittest.py changes themselves, if we can figure out a way to add our test to that config["all_mochitest_suites"] list within thunderbird_extra.py, I'm all for it. Even better (and this is will take some work), we could add support for adding to the Mozharness config path to desktop_unittest.py like we do for fx_desktop_build.py. This would let us move thunderbird_extra.py to comm/mozharness/testing (or something like that). - Update taskcluster/taskgraph/transforms/tests.py test_description_schema to allow for config-paths key. (See https://hg.mozilla.org/comm-central/file/tip/taskcluster/ci/build/linux.yml#l18) - Update comm/taskcluster/ci/test/tests.yml with config-path set to "comm/mozharness" - Update taskcluster/transforms/job/mozharness_test.py to pass --config-path along as MOZHARNESS_CONFIG_PATHS like https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/taskgraph/transforms/job/mozharness.py#l181 - Update taskcluster/scripts/tester/test-{linux,macosx}.sh to recognize $MOZHARNESS_CONFIG_PATHS env variable like https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/scripts/builder/build-linux.sh#l15 and https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/scripts/builder/build-linux.sh#l89 Maybe more... Like I said, that last part is a lot of work, and probably not necessary at this time.

Comment on attachment 9024581 [details] [diff] [review] 1506715-mochitest-1.diff Review of attachment 9024581 [details] [diff] [review]: ----------------------------------------------------------------- The mochitest harness changes are fine, but I think you need a clearer separation of thunderbird from the mozharness configuration. Ideally, there shouldn't be thunderbird mozharness configuration in mozilla-central.

Rob, can you take this from here? I'll tidy up and post the comm-central side of things for you to look at too.

Separating out the mochitest pieces from the mozharness ones.

There's something weird going on with that version 2 patch. Most chunks appear in it twice for some reason. Could we at least land this part (same as before without the mozharness bits), so that our developers can run tests locally?

Patch failed to apply. patching file testing/mochitest/browser-test.js Hunk #5 FAILED at 906 1 out of 5 hunks FAILED -- saving rejects to file testing/mochitest/browser-test.js.rej patch failed, unable to continue (try -v) patch failed, rejects left in working directory errors during apply, please fix and qrefresh 1506715-mochitest-3.diff

Oh damn, I forgot that patch had bitrotted. Will fix.

Let's try that again.

Pushed by btara@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/adcf2d91453e Adapt Mochitest so it can run on Thunderbird r=darktrojan

Spun off bug 1506715 as the mozharness configuration is turning into it's own task. Geoff, giving this back to you.

In that case, we're done here.

I'm pretty sure you meant to say bug 1517298 in that last comment. :)