Description: Test case fails with "AssertionError" Steps: 1. Update a flame device to 20151001030229 2. Run this test case http://lxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/functional/dialer/test_call_log_groups.py Actual result: "AssertionError" Expected result: Test completes successfully. Repro Rate: Reproduced via Jenkins (0/5 passing) http://jenkins1.qa.scl3.mozilla.com/job/flame-kk.ui.adhoc.bitbar/265/HTML_Report/ Traceback (most recent call last): File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/.env/lib/python2.7/site-packages/marionette_client-0.17-py2.7.egg/marionette/marionette_test.py", line 296, in run testMethod() File "/var/lib/jenkins/jobs/flame-kk.ui.adhoc.bitbar/workspace/tests/python/gaia-ui-tests/gaiatest/tests/functional/dialer/test_call_log_groups.py", line 52, in test_call_log_groups self.assertEqual('{:%m/%d/%Y}'.format(self.past_date_1), group_calls[2].header_text) AssertionError: '12/01/2014' != u'12/01/14'

This started failing because of bug 1206306.

Comment on attachment 8668932 [details] [gaia] mwargers:1210466 > mozilla-b2g:master This fixes the failure for me. I also had to change the timezone for this test, otherwise I would get this failure: TEST-UNEXPECTED-FAIL | test_call_log_groups.py TestCallLogGroups.test_call_log_groups | AssertionError: u'12/01/14' != u'11/30/14' - 12/01/14 ? ^ - + 11/30/14 ? ^ + Traceback (most recent call last): File "/Users/mwargers/.virtualenvs/zapion_test2/lib/python2.7/site-packages/marionette_client-0.17-py2.7.egg/marionette/marionette_test.py", line 296, in run testMethod() File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/tests/functional/dialer/test_call_log_groups.py", line 53, in test_call_log_groups self.assertEqual(u'{:%m/%d/%y}'.format(self.past_date_1), group_calls[2].header_text) TEST-INFO took 93998ms Not exactly sure why that is needed, but I guess it has something to do with the usage of UTC, which is then converted to a certain time on the phone.

Why this error did show up for me when I was working on the original patch? I would have investigate on my own. Sorry for that, Martijn!

Zibi, no problem. Gaia UI tests are not running on Treeherder, so there is no way you could have known. This has been going on for a while and is not an ideal situation. In the next quarter we want to get it back to report to Treeherder.

Comment on attachment 8668932 [details] [gaia] mwargers:1210466 > mozilla-b2g:master git grep '{:%m/%d/%Y}' showed only the 2 lines you changed, which is good. I agree with you regarding the timezone change. It seems related to bug 1206306#c8> In order to make the test less error-prone I think we can force the timezone to UTC.

I removed this part: - self.data_layer.set_setting('time.timezone', 'Europe/Istanbul') And started an adhoc job to see if this works fine in the Mountain View lab: http://jenkins1.qa.scl3.mozilla.com/view/Mozilla%20Lab/job/flame-kk.ui.adhoc/894/ I'm still surprised that I need this setting locally to get this test passing. Setting the timezone to Europe/Paris also makes the test pass for me, but setting the timezone Europe/Amsterdam, I get the same failure.

Right adhoc job this time: http://jenkins1.qa.scl3.mozilla.com/view/Mozilla%20Lab/job/flame-kk.ui.adhoc/895/

Ok, the adhoc job is passing.

Comment on attachment 8668932 [details] [gaia] mwargers:1210466 > mozilla-b2g:master LGTM

Merged: https://github.com/mozilla-b2g/gaia/commit/7ff515a65174cebf484cf0e17af82144103a73f0 Zibi, do you have an idea why I had to set the time.timezone setting to 'Europe/Istanbul' or 'Europe/Paris' to get not the failure as in comment 4? Setting the time.timezone setting to 'Europe/Amsterdam' also gives me that failure.

Zibi, see question in comment 12.

My guess is that your test is timezone dependent so when it's run in one timezone it may show one day, and in another, another. You should force all assertions to either be timezone dependent on both sides, or timezone independent on both sides (forced to UTC).

I don't see how this test is timezone dependant and even if it was how it can be that it works and not works in the cases mentioned in comment 14. There should have been no difference between 'Europe/Paris' and 'Europe/Amsterdam' timezone.