Raphael Pierzina [:raphael] UTC+01:00 Reporter
	Description • 6 years ago

The telemetry-tests-client suite uses firefox-puppeteer for browser UI automation. We have observed intermittent failures on CI caused by NoSuchWindowException errors when calling TabBar.open_tab().

https://searchfox.org/mozilla-central/source/testing/marionette/puppeteer/firefox/firefox_puppeteer/ui/browser/tabbar.py#113

https://treeherder.mozilla.org/#/jobs?repo=try&revision=61bb5e39ae45d0aa925cae2e393fddbc2e91ce5d

	Henrik Skupin [:whimboo][⌚️UTC+2]
	Comment 1 • 6 years ago

FYI you won't need this method anymore once bug 1504756 has been landed. I hope that this will happen this week (as no other blockers turn up).

Here the current trace log:

https://treeherder.mozilla.org/logviewer.html#?job_id=213893268&repo=try&lineNumber=2713-2727

> [task 2018-11-26T18:38:00.783Z] 18:38:00     INFO - TEST-UNEXPECTED-ERROR | telemetry/marionette/tests/client/test_search_counts_across_sessions.py TestSearchCounts.test_search_counts | NoSuchWindowException: No window found for '1'
> [task 2018-11-26T18:38:00.783Z] 18:38:00     INFO - Traceback (most recent call last):
> [task 2018-11-26T18:38:00.784Z] 18:38:00     INFO -   File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 159, in run
> [task 2018-11-26T18:38:00.785Z] 18:38:00     INFO -     testMethod()
> [task 2018-11-26T18:38:00.786Z] 18:38:00     INFO -   File "/builds/worker/workspace/build/tests/telemetry/marionette/tests/client/test_search_counts_across_sessions.py", line 67, in test_search_counts
> [task 2018-11-26T18:38:00.786Z] 18:38:00     INFO -     self.search_in_new_tab("mozilla firefox")
> [task 2018-11-26T18:38:00.788Z] 18:38:00     INFO -   File "/builds/worker/workspace/build/tests/telemetry/marionette/tests/client/test_search_counts_across_sessions.py", line 49, in search_in_new_tab
> [task 2018-11-26T18:38:00.789Z] 18:38:00     INFO -     new_tab = self.browser.tabbar.open_tab()
> [task 2018-11-26T18:38:00.790Z] 18:38:00     INFO -   File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/firefox_puppeteer/ui/browser/window.py", line 140, in tabbar
> [task 2018-11-26T18:38:00.791Z] 18:38:00     INFO -     self.switch_to()
> [task 2018-11-26T18:38:00.792Z] 18:38:00     INFO -   File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/firefox_puppeteer/ui/windows.py", line 433, in switch_to
> [task 2018-11-26T18:38:00.793Z] 18:38:00     INFO -     self._windows.switch_to(self.handle)
> [task 2018-11-26T18:38:00.794Z] 18:38:00     INFO -   File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/firefox_puppeteer/ui/windows.py", line 189, in switch_to
> [task 2018-11-26T18:38:00.795Z] 18:38:00     INFO -     .format(target))

Note that the problem is NOT in `tabbar.py` but `browser/window.py`, when calling `switch_to()`, which means that `open_tab()` returns too early.

Raphael, can you please provide a minimal testcase to reproduce this problem?

	Raphael Pierzina [:raphael] UTC+01:00 Reporter
	Comment 2 • 6 years ago

Does this test from mozilla-central work for you as a minimal testcase? It is functional, but currently not included in the telemetry-tests-client suite due to the described error. It fails when calling `self.search_in_new_tab("mozilla firefox")` at the start of the test before it can perform any assertions:

```
def search_in_new_tab(self, text):
    """Open a new tab and perform a search via the browser's location
    bar, then close the new tab.
    """
    with self.marionette.using_context(self.marionette.CONTEXT_CHROME):
        new_tab = self.browser.tabbar.open_tab()
        self.browser.tabbar.switch_to(new_tab)
        self.browser.navbar.locationbar.load_url(text)
        new_tab.close()
```

https://searchfox.org/mozilla-central/source/toolkit/components/telemetry/tests/marionette/tests/client/test_search_counts_across_sessions.py#67

	Henrik Skupin [:whimboo][⌚️UTC+2]
	Comment 3 • 6 years ago

Here a minimized testcase which shows a failure for me but it different to yours. Here I see it when closing the tab, and that happens because the window handle has been changed due to performing a search in the new tab. This causes a process move for teh current tab. See bug 1311041 for details. So using `self.marionette.close()` should fix it.

It would be great if you could modify the test so that it show up the correct error. You can use the `--run-until-failure` argument for mach or marionette CLI, to let the test repeatedly run until it fails or reaches 30 iterations.


from firefox_puppeteer import PuppeteerMixin
from marionette_harness import MarionetteTestCase


class Test(PuppeteerMixin, MarionetteTestCase):

    def test_new_tab(self):
        with self.marionette.using_context(self.marionette.CONTEXT_CHROME):
            new_tab = self.browser.tabbar.open_tab()
            self.browser.tabbar.switch_to(new_tab)
            self.browser.navbar.locationbar.load_url("awe")
            new_tab.close()

	Raphael Pierzina [:raphael] UTC+01:00 Reporter
	Comment 5 • 6 years ago

I have not been able to reproduce the NoSuchWindowException on open_tab() by modifying this minimal test case. Instead the error appears to always happen on new_tab.close().

	Henrik Skupin [:whimboo][⌚️UTC+2]
	Comment 6 • 6 years ago

This is the minimal testcase as authored by Raphael in the phabricator revision which I just closed.