Follow-up from bug 1204888. This query: http://mxr.mozilla.org/gaia/search?string=displayed_app.name&find=.py Only gives 22 instances left of displayed_app.name. Those need to disappear too. Some of them might be difficult, so it might be good to only try to fix the ones that are reasonable doable here and fix the rest in a separate bug/pull request. Manel, feel free to take this bug or else I'll pick it up at some point.

(In reply to Martijn Wargers [:mwargers] (QA) from comment #0) > Follow-up from bug 1204888. > This query: > http://mxr.mozilla.org/gaia/search?string=displayed_app.name&find=.py > Only gives 22 instances left of displayed_app.name. > Those need to disappear too. > > Some of them might be difficult, so it might be good to only try to fix the > ones that are reasonable doable here and fix the rest in a separate bug/pull > request. > > Manel, feel free to take this bug or else I'll pick it up at some point. I would take it Martijin, thanks

When I was trying out this tests (changed in the PR) I got some failure which is not related to our Work: /functional/cards_view/test_cards_view_with_two_apps.py: https://pastebin.mozilla.org/8847472 functional/marketplace/test_marketplace_packaged_app.py: https://pastebin.mozilla.org/8847473 graphics/test_cards_view_with_two_apps.py : https://pastebin.mozilla.org/8847474 graphics/test_marketplace_download_execute_map_app.py : https://pastebin.mozilla.org/8847475 tests/unit/test_cleanup_gaia.py: https://pastebin.mozilla.org/8847479 for the accessibility test I would yzen could help us: /accessibility/system/test_a11y_cards_view.py : https://pastebin.mozilla.org/8847476 accessibility/phone/test_a11y_phone_call_log.py: https://pastebin.mozilla.org/8847477 accessibility/ftu/test_a11y_ftu.py and /accessibility/ftu/test_a11y_ftu_desktopb2g.py : https://pastebin.mozilla.org/8847478

(In reply to Manel Rhaiem [:manel] from comment #3) > When I was trying out this tests (changed in the PR) I got some failure > which is not related to our Work: > > /functional/cards_view/test_cards_view_with_two_apps.py: > https://pastebin.mozilla.org/8847472 See http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/functional/cards_view/manifest.ini#10 Disabled because of bug 1172167. > functional/marketplace/test_marketplace_packaged_app.py: > https://pastebin.mozilla.org/8847473 That is because you forgot to unremove the - expected_name = self.name line in homescreen/app.py in your pull request. > graphics/test_cards_view_with_two_apps.py : > https://pastebin.mozilla.org/8847474 This is also disabled, also because of bug 1172167. > graphics/test_marketplace_download_execute_map_app.py : > https://pastebin.mozilla.org/8847475 That is also because you forgot to unremove the - expected_name = self.name line in homescreen/app.py in your pull request. > tests/unit/test_cleanup_gaia.py: https://pastebin.mozilla.org/8847479 Disabled: http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/unit/manifest.ini#11 That is due to bug 1196358, likely to be fixed soon.

> > for the accessibility test I would yzen could help us: > /accessibility/system/test_a11y_cards_view.py : > https://pastebin.mozilla.org/8847476 That might also be due to bug 1172167. But it's not disabled. > accessibility/phone/test_a11y_phone_call_log.py: > https://pastebin.mozilla.org/8847477 That is failing because #call-log-icon-close element can't be found, which doesn't seem to exist indeed. http://mxr.mozilla.org/gaia/search?string=call-log-icon-close This test is not disabled: http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/accessibility/phone/manifest.ini#11 > accessibility/ftu/test_a11y_ftu.py and > /accessibility/ftu/test_a11y_ftu_desktopb2g.py : > https://pastebin.mozilla.org/8847478 I suspect these tests haven't been updated lately. Yura, are the accessibility tests running anywhere? Or are they not being used anymore?

> graphics/test_cards_view_with_two_apps.py : > https://pastebin.mozilla.org/8847474 > graphics/test_marketplace_download_execute_map_app.py : > https://pastebin.mozilla.org/8847475 > tests/unit/test_cleanup_gaia.py: https://pastebin.mozilla.org/8847479 BTW, in general, you don't have to check that graphics tests pass, since they are outside of the regular test suite. But in case of graphics/test_marketplace_download_execute_map_app.py, it looks like the name has to be changed.

(In reply to Martijn Wargers [:mwargers] (QA) from comment #5) > > > > for the accessibility test I would yzen could help us: > > /accessibility/system/test_a11y_cards_view.py : > > https://pastebin.mozilla.org/8847476 > > That might also be due to bug 1172167. But it's not disabled. > > > accessibility/phone/test_a11y_phone_call_log.py: > > https://pastebin.mozilla.org/8847477 > > That is failing because #call-log-icon-close element can't be found, which > doesn't seem to exist indeed. > http://mxr.mozilla.org/gaia/search?string=call-log-icon-close > This test is not disabled: > http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/ > accessibility/phone/manifest.ini#11 > > > accessibility/ftu/test_a11y_ftu.py and > > /accessibility/ftu/test_a11y_ftu_desktopb2g.py : > > https://pastebin.mozilla.org/8847478 > > I suspect these tests haven't been updated lately. > > Yura, are the accessibility tests running anywhere? Or are they not being > used anymore? It looks like they do not run on treehereder. Because test_a11y_cards_view.py would then be failing for current master. The test itself doesn't look out of date. I investigated a bit and it looks like the cards view accessibility regressed in itself. So the trace you provided doesn't seem to be a result of your changes and rather an actual regression in cards view. I opened bug 1208490 to track the actual issue.

Yura, the Gip and Gip accessibility tests are disabled for a few months already on Treeherder. It had to do with too many intermittent failures and with the move over to MarionetteJS. I guess the accessibility tests could run as part of Jenkins perhaps, but we/I should discuss with Johan about this. We from Gaia UI automation are not maintaining the accessibility tests at all (it was specifically mentioned to me to not do that). I think the accessibility tests should definitely be run periodically, otherwise they are not valuable and are bound to become outdated at some point.

(In reply to Martijn Wargers [:mwargers] (QA) from comment #8) > Yura, the Gip and Gip accessibility tests are disabled for a few months > already on Treeherder. It had to do with too many intermittent failures and > with the move over to MarionetteJS. > I guess the accessibility tests could run as part of Jenkins perhaps, but > we/I should discuss with Johan about this. We from Gaia UI automation are > not maintaining the accessibility tests at all (it was specifically > mentioned to me to not do that). > I think the accessibility tests should definitely be run periodically, > otherwise they are not valuable and are bound to become outdated at some > point. I think in the long run (ideally) those tests should just be ported (if necessary, otherwise we will just enable a11y flag for Gij for all of them once they can pass) to Gij, since they really only differ in the type of marionette client that is used.

Yes, makes sense. I guess it's better to not touch the accessibility tests at all then, since the plan is to port them over to Gij.

Moreover, we're planning to run the tests on TaskCluster (and report the results on Treeherder). I'd prefer to not create a new job, unless we want to roll back to Jenkins.

Comment on attachment 8664841 [details] [gaia] mermi:bug-1207098 > mozilla-b2g:master I maded the comments in the pull request.

Marking WONTFIX, sorry for the bug spam. If somebody still wants to work on this, please file a new bug for it.