https://tbpl.mozilla.org/php/getParsedLog.php?id=34518484&tree=Ash&full=1#error4 16:00:18 INFO - 6 INFO TEST-PASS | testBookmarksPanel | Checking that the correct tab is displayed - The BOOKMARKS tab is displayed 16:00:18 INFO - 7 INFO TEST-PASS | testBookmarksPanel | Checking that about:firefox displayed as a bookmark - about:firefox displayed 16:00:18 INFO - 8 INFO TEST-PASS | testBookmarksPanel | waiting for urlbar text to gain focus - urlbar text gained focus 16:00:18 INFO - 9 INFO TEST-PASS | testBookmarksPanel | Checking that the correct tab is displayed - The BOOKMARKS tab is displayed 16:00:18 INFO - 10 INFO TEST-PASS | testBookmarksPanel | Checking that http://support.mozilla.org/en-US/products/mobile displayed as a bookmark - http://support.mozilla.org/en-US/products/mobile displayed 16:00:18 INFO - 11 INFO TEST-PASS | testBookmarksPanel | waiting for urlbar text to gain focus - urlbar text gained focus 16:00:18 INFO - 12 INFO TEST-PASS | testBookmarksPanel | Checking that the correct tab is displayed - The BOOKMARKS tab is displayed 16:00:18 INFO - 13 INFO TEST-PASS | testBookmarksPanel | Checking that https://addons.mozilla.org/en-US/android/ displayed as a bookmark - https://addons.mozilla.org/en-US/android/ displayed 16:00:18 INFO - 14 INFO TEST-PASS | testBookmarksPanel | waiting for urlbar text to gain focus - urlbar text gained focus 16:00:18 INFO - 15 INFO TEST-PASS | testBookmarksPanel | Checking that the correct tab is displayed - The BOOKMARKS tab is displayed 16:00:18 INFO - 16 INFO TEST-UNEXPECTED-FAIL | testBookmarksPanel | Checking that the context menu option is present - Open in New Tab is present 16:00:18 INFO - Exception caught during test! 16:00:18 INFO - junit.framework.AssertionFailedError: 16 INFO TEST-UNEXPECTED-FAIL | testBookmarksPanel | Checking that the context menu option is present - Open in New Tab is present 16:00:18 INFO - at junit.framework.Assert.fail(Assert.java:47) 16:00:18 INFO - at org.mozilla.gecko.FennecMochitestAssert._logMochitestResult(FennecMochitestAssert.java:110) 16:00:18 INFO - at org.mozilla.gecko.FennecMochitestAssert.ok(FennecMochitestAssert.java:139) 16:00:18 INFO - at org.mozilla.gecko.tests.testBookmarksPanel.testBookmarksPanel(testBookmarksPanel.java:29) 16:00:18 INFO - at java.lang.reflect.Method.invokeNative(Native Method) 16:00:18 INFO - at java.lang.reflect.Method.invoke(Method.java:507) 16:00:18 INFO - at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204) 16:00:18 INFO - at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194) 16:00:18 INFO - at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186) 16:00:18 INFO - at org.mozilla.gecko.tests.BaseTest.runTest(BaseTest.java:150) 16:00:18 INFO - at junit.framework.TestCase.runBare(TestCase.java:127) 16:00:18 INFO - at junit.framework.TestResult$1.protect(TestResult.java:106) 16:00:18 INFO - at junit.framework.TestResult.runProtected(TestResult.java:124) 16:00:18 INFO - at junit.framework.TestResult.run(TestResult.java:109) 16:00:18 INFO - at junit.framework.TestCase.run(TestCase.java:118) 16:00:18 INFO - at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169) 16:00:18 INFO - at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154) 16:00:18 INFO - at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:529) 16:00:18 INFO - at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1448) 16:00:18 INFO - 17 INFO TEST-UNEXPECTED-FAIL | testBookmarksPanel | Exception caught - junit.framework.AssertionFailedError: 16 INFO TEST-UNEXPECTED-FAIL | testBookmarksPanel | Checking that the context menu option is present - Open in New Tab is present 16:00:18 INFO - 18 INFO TEST-END | testBookmarksPanel | finished in 115138ms 16:00:18 INFO - 19 INFO TEST-START | Shutdown 16:00:18 INFO - 20 INFO Passed: 14 16:00:18 WARNING - 21 INFO Failed: 2

Confirmed that this test still fails if enabled: https://tbpl.mozilla.org/php/getParsedLog.php?id=41585120&tree=Ash&full=1#error2

(In reply to Geoff Brown [:gbrown] from comment #3) > Confirmed that this test still fails if enabled: It appears to be a different issue, however. CC'ing snorp as he seems to touch most of the process crashed robocop bugs.

The crash in comment 3 is bug 1025968. But I may have found a way to avoid it...

A lot of the problems I saw in testBookmarksPanel on 2.3 seemed to happen in getDisplayedBookmark, called from isBookmarkDisplayed (inside a BooleanTest). That path is called a lot in this test, and each getDisplayedBookmark calls openAboutHomeTab(BOOKMARKS) even though the whole test pretty much happens on that tab. In this patch, I pull openAboutHomeTab out of getDisplayedBookmark so we can call it just once at the beginning of the test and avoid a whole lot of re-selection. I removed the BooleanTest from isBookmarkDisplayed -- it does not seem to be necessary. And I changed an extra-long long click (which was not always working on 2.3) into a regular long click + a single retry if the original long click fails. The result seems reliable: https://tbpl.mozilla.org/?tree=Try&rev=ccc222a0fe0a

Comment on attachment 8442188 [details] [diff] [review] simplify testBookmarksPanel Review of attachment 8442188 [details] [diff] [review]: ----------------------------------------------------------------- ::: mobile/android/base/tests/AboutHomeTest.java @@ +58,5 @@ > return null; > } > > // Returns true if the bookmark is displayed in the bookmarks tab, false otherwise - does not check in folders > protected void isBookmarkDisplayed(final String url) { nit: is* should return a boolean. This would better be called assertBookmarkDisplayed (I know this change is unrelated). @@ +78,5 @@ > } > > // Opens the bookmark context menu by long-tapping on it > protected void openBookmarkContextMenu(String url) { > + waitForText(url); Is this necessary? We scroll the ListView in getDisplayedBookmark so we should get to see all of the elements. @@ +83,4 @@ > View bookmark = getDisplayedBookmark(url); > if (bookmark != null) { > mSolo.waitForView(bookmark); > + mSolo.clickLongOnView(bookmark); When might this fail? @@ -97,5 @@ > } > > // @return the View associated with bookmark for the provided url or null if the link is not bookmarked > protected View getDisplayedBookmark(String url) { > - openAboutHomeTab(AboutHomeTabs.BOOKMARKS); I'm not sure that all of the places where `getDisplayedBookmark` is called explicitly open the BOOKMARKS Tab. e.g. AboutHomeTest.loadBookmark -> testBookmark.runAboutHomeTest. Did you look to make sure the bookmarks tab was opened for all tests? It may also be a good idea to add an assertion that the BOOKMARKS tab is actually open.

I can never seem to find time for this. testBookmarksPanel remains disabled on 2.3 and 4.3.

Android 2.3 is no longer supported in Firefox 48+. Test manifests were updated in bug 1251013.