Closed Bug 1338040 Opened 8 years ago Closed 6 years ago

Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past | AssertionError: 0 != 1

Categories

(Testing :: Marionette Client and Harness, defect, P1)

Product:
Testing
Component:
Marionette Client and Harness
Version:
Version 3
Platform:
All
Android
Type:
defect

Tracking

(firefox67 fixed, firefox68 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla68
Tracking Flags:
Tracking Status
firefox67 --- fixed
firefox68 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: whimboo)

References

Details

(Keywords: intermittent-failure, mobile, Whiteboard: [stockwell disable-recommended])

Attachments

(1 file)

Bug 1338040 - [marionette] Use a cookie expired a day ago for test_adding_a_cookie_that_expired_in_the_past.
(deleted), text/x-phabricator-request
Details
OS: Unspecified → Android
Hardware: Unspecified → ARM
[task 2017-02-09T02:39:08.880335Z] 02:39:08 WARNING - TEST-UNEXPECTED-FAIL | test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past | AssertionError: 0 != 1 [task 2017-02-09T02:39:08.880725Z] 02:39:08 INFO - Traceback (most recent call last): [task 2017-02-09T02:39:08.881063Z] 02:39:08 INFO - File "/home/worker/workspace/build/venv/local/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 166, in run [task 2017-02-09T02:39:08.881297Z] 02:39:08 INFO - testMethod() [task 2017-02-09T02:39:08.881693Z] 02:39:08 INFO - File "/home/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py", line 38, in test_adding_a_cookie_that_expired_in_the_past [task 2017-02-09T02:39:08.881867Z] 02:39:08 INFO - self.assertEquals(0, len(cookies))
The tests are probably failing because the result of time.gmtime() is dependent on system clock. There is a slim chance that NTP adjusts the time between the creation and the addition of the cookie. A solution for this would be to set the expiration time a bit further in the past for the test.
https://wiki.mozilla.org/Bugmasters#Intermittent_Test_Failure_Cleanup
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → INCOMPLETE
Reopening since it's happening again. Recent log: https://treeherder.mozilla.org/logviewer.html#?job_id=157285299&repo=mozilla-central&lineNumber=2006
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Status: REOPENED → NEW
Priority: -- → P5
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
Status: NEW → RESOLVED
Closed: 7 years ago → 7 years ago
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
Status: REOPENED → RESOLVED
Closed: 7 years ago → 7 years ago
Resolution: --- → INCOMPLETE
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
Status: REOPENED → RESOLVED
Closed: 7 years ago → 6 years ago
Resolution: --- → INCOMPLETE
Fail reappeared on inbound: https://treeherder.mozilla.org/logviewer.html#?job_id=186344931&repo=mozilla-inbound&lineNumber=1599
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
Status: REOPENED → RESOLVED
Closed: 6 years ago → 6 years ago
Resolution: --- → INCOMPLETE
Recent failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=197702810&repo=autoland&lineNumber=1591 [task 2018-09-05T21:31:15.674Z] 21:31:15 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past [task 2018-09-05T21:31:21.236Z] 21:31:21 WARNING - TEST-UNEXPECTED-FAIL | testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past | AssertionError: 0 != 1 [task 2018-09-05T21:31:21.237Z] 21:31:21 INFO - Traceback (most recent call last): [task 2018-09-05T21:31:21.238Z] 21:31:21 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 159, in run [task 2018-09-05T21:31:21.238Z] 21:31:21 INFO - testMethod() [task 2018-09-05T21:31:21.239Z] 21:31:21 INFO - File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py", line 40, in test_adding_a_cookie_that_expired_in_the_past [task 2018-09-05T21:31:21.239Z] 21:31:21 INFO - self.assertEquals(0, len(cookies)) [task 2018-09-05T21:31:21.240Z] 21:31:21 INFO - TEST-INFO took 5563ms [task 2018-09-05T21:31:21.577Z] 21:31:21 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py CookieTest.test_chrome_error
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
Status: REOPENED → RESOLVED
Closed: 6 years ago → 6 years ago
Resolution: --- → INCOMPLETE

Recent log failure: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=232496678&repo=autoland&lineNumber=2653

[task 2019-03-07T19:54:43.887Z] 19:54:43 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past
[task 2019-03-07T19:54:44.240Z] 19:54:44 WARNING - TEST-UNEXPECTED-FAIL | testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past | AssertionError: 0 != 1
[task 2019-03-07T19:54:44.240Z] 19:54:44 INFO - Traceback (most recent call last):
[task 2019-03-07T19:54:44.241Z] 19:54:44 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 159, in run
[task 2019-03-07T19:54:44.241Z] 19:54:44 INFO - testMethod()
[task 2019-03-07T19:54:44.241Z] 19:54:44 INFO - File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py", line 42, in test_adding_a_cookie_that_expired_in_the_past
[task 2019-03-07T19:54:44.242Z] 19:54:44 INFO - self.assertEquals(0, len(cookies))
[task 2019-03-07T19:54:44.242Z] 19:54:44 INFO - TEST-INFO took 353ms

Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---

Geoff tried to fix that with bug 1526913 but as it looks like we can still hit this failure.

Blocks: 1500509
Depends on: 1526913
Hardware: ARM → All
Summary: Intermittent test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past | AssertionError: 0 != 1 → Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_cookies.py CookieTest.test_adding_a_cookie_that_expired_in_the_past | AssertionError: 0 != 1

This bug has failed 57 times in the last 7 days. Occurs on android-em-7-0-x86_64 opt and debug build types.

Recent log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=241181373&repo=autoland&lineNumber=2681

Reviewing recent failures, almost all have android-performance.log artifacts corresponding to bug 1545308.

Depends on: 1545308

Geoff, any idea when the situation on packet.net will become better? Otherwise we might just extend the time for this test to 1h or so, so that slowness doesn't affect us this much.

Flags: needinfo?(gbrown)
Keywords: mobile

I think there is a good chance we can resolve it soon. I've nudged :coop, https://bugzilla.mozilla.org/show_bug.cgi?id=1545308#c2.

Flags: needinfo?(gbrown)

To make the test more robust lets use a cookie which already expired a day ago and not just some seconds before.

Here a try build:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=831c0dde08f3416b84b3c8289f1495f8add926da

Assignee: nobody → hskupin
Status: REOPENED → ASSIGNED
Priority: P5 → P1
Pushed by hskupin@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/91925dfb4532 [marionette] Use a cookie expired a day ago for test_adding_a_cookie_that_expired_in_the_past. r=ato

https://hg.mozilla.org/mozilla-central/rev/91925dfb4532

Status: ASSIGNED → RESOLVED
Closed: 6 years ago → 6 years ago
status-firefox68: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
Product: Testing → Remote Protocol
Moving bug to Testing::Marionette Client and Harness component per bug 1815831.
Component: Marionette → Marionette Client and Harness
Product: Remote Protocol → Testing
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: