Once we use the patch of Bug 950673, some tests will be failed. Always failed: apps/system/test/marionette/notification_get_test.js:247 apps/system/test/marionette/edges_gesture_test.js:30 apps/music/test/marionette/Player_test.js:103 apps/calendar/test/marionette/month_view_test.js:16 apps/calendar/test/marionette/toggle_calendar_test.js:13 apps/calendar/test/marionette/day_view_test.js:25 apps/calendar/test/marionette/create_event_test.js:22 Intermittent failed: apps/system/test/marionette/fullscreen_statusbar_test.js:30 apps/search/test/marionette/places_search_test.js: apps/search/test/marionette/app_search_test.js:15 apps/homescreen/test/marionette/install_bookmark_test.js:43 We could refer to https://travis-ci.org/evanxd/gaia/builds/23940818.

Currently, we need to disable the below tests with the https://github.com/evanxd/marionette-apps/commit/2665d203c85eb9b02a000ac9716edcc15b274ee3 marionette-apps change. 1. apps/calendar/test/marionette/create_event_test.js 2. apps/calendar/test/marionette/day_view_test.js 3. apps/calendar/test/marionette/month_view_test.js 4. apps/calendar/test/marionette/toggle_calendar_test.js 5. apps/homescreen/test/marionette/install_bookmark_test.js 6. apps/music/test/marionette/Player_test.js 7. apps/search/test/marionette/app_search_test.js 8. apps/search/test/marionette/places_search_test.js 9. apps/system/test/marionette/desktop_notification_test.js 10. apps/system/test/marionette/edges_gesture_test.js 11. apps/system/test/marionette/fullscreen_statusbar_test.js 12. apps/system/test/marionette/notification_test.js 13. apps/system/test/marionette/notification_get_test.js

And all tests will be passed after we do the marionette-apps patch at Comment 1. See the travis job at https://travis-ci.org/evanxd/gaia/builds/24507793.

After we fix this bug, we should find a long term solution to deal with changes of System app for the marionette-apps module.

We will disable the below tests in Bug 950673, because we will use updated marionette-apps module and the logic of switchToApp method is changed. home2: "dev_apps/home2/test/marionette/bookmark_test.js" calendar: "apps/calendar/test/marionette/create_event_test.js" "apps/calendar/test/marionette/day_view_test.js" "apps/calendar/test/marionette/month_view_test.js" "apps/calendar/test/marionette/toggle_calendar_test.js" "apps/homescreen/test/marionette/install_bookmark_test.js" music: "apps/music/test/marionette/Player_test.js" search: "apps/search/test/marionette/app_search_test.js" "apps/search/test/marionette/places_search_test.js" system: "apps/system/test/marionette/desktop_notification_test.js" "apps/system/test/marionette/edges_gesture_test.js" "apps/system/test/marionette/fullscreen_statusbar_test.js" "apps/system/test/marionette/notification_events_test.js" "apps/system/test/marionette/notification_test.js" "apps/system/test/marionette/notification_get_test.js" We will fix them here.

The version of marionette-apps we will use in Bug 950673 is v0.3.5.

Sorry, it is also about homescreen app. homescreen: "apps/homescreen/test/marionette/install_bookmark_test.js"

So why is the switchToApp logic different now? I see some apps with: client.switchToFrame(); client.apps.(url); What is so broken about this?

Hi Kevin, Currently, `switchToApp` method wait for the app's frame is on the `transition-state="opened"` state and the class list included `render`. It means the app is already rendered and show on the screen, and then the app is ready to be controlled by marionette. We could refer to Bug 1003788. After we have the updated `switchToApp` method, we could not do the below things. 1. Do not use the `switchToApp` method to switch to keyboard frame. 2. Do not use the `switchToApp` method to switch to homescreen frame, when we already launched some app. And the above two wrong cases is in the gaia code base. For example, this line https://github.com/mozilla-b2g/gaia/blob/master/dev_apps/home2/test/marionette/bookmark_test.js#L52 should do `client.switchToFrame(home2Frame)` to instead of `client.apps.switchToApp(Home2.URL)`. For "apps/calendar/test/marionette/create_event_test.js", "apps/calendar/test/marionette/day_view_test.js", "apps/calendar/test/marionette/month_view_test.js", "apps/calendar/test/marionette/toggle_calendar_test.js", this line https://github.com/mozilla-b2g/gaia/blob/master/apps/calendar/test/marionette/lib/calendar.js#L280 should to `client.switchToFrame(keyboardFrame)` to instead of `client.apps.switchToApp('app://keyboard.gaiamobile.org');`. We need to fix the wrong usage of marionette-apps in gaia case by case.

I think this (and the patch to marionette-apps) are invalid and that the issue has more to do with the system app page loading semantics than it does correctness of integration tests...

I don't really see how there can be a problem with loading semantics if we can load apps, and the phone is functional. It seems like we just need to find a better fix for marionette-apps so we don't have to patch the tests.

(In reply to Gareth Aye [:gaye] on PTO until 06/02 from comment #9) > I think this (and the patch to marionette-apps) are invalid and that the > issue has more to do with the system app page loading semantics than it does > correctness of integration tests... Well, system app is not existing to provide any support to a not-correctly-implemented test library. As somebody's note: there should be more communication made before people invented this library between library author and system app owner.

Mass modify - set status-b2g-v2.0 fixed for fixed bugs under vertical homescreen dependency tree.