Closed Bug 1665377 Opened 4 years ago Closed 4 years ago

Perma Beta TEST-UNEXPECTED-FAIL | testing/marionette/harness/marionette_harness/tests/unit/test_navigation.py TestTLSNavigation.test_deactivation | AssertionError: InsecureCertificateException not raised when Gecko 82 merges to Beta on 2020-09-21

Categories

(Remote Protocol :: Marionette, defect, P5)

Product:
Remote Protocol
Component:
Marionette
Version:
Firefox 82
Platform:
All
Linux
Type:
defect

Tracking

(firefox-esr68 unaffected, firefox-esr78 unaffected, firefox80 unaffected, firefox81 unaffected, firefox82+ fixed)

Status:
RESOLVED FIXED
Milestone:
82 Branch
Tracking Flags:
Tracking Status
firefox-esr68 --- unaffected
firefox-esr78 --- unaffected
firefox80 --- unaffected
firefox81 --- unaffected
firefox82 + fixed

People

(Reporter: malexandru, Assigned: impossibus)

References

(Regression)

Details

(Keywords: regression)

Central as Beta Simulation:
https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=b39448e41daa6e1adea7fa6acf6dd2fc902dc654&searchStr=marionette&selectedTaskRun=eOv-lon7S-WQv-brx6KLrw.0

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=315830850&repo=try&lineNumber=20315

Raw log: https://firefoxci.taskcluster-artifacts.net/eOv-lon7S-WQv-brx6KLrw/0/public/logs/live_backing.log

[task 2020-09-16T12:28:27.974Z] 12:28:27     INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_navigation.py TestTLSNavigation.test_deactivation
[task 2020-09-16T12:28:27.975Z] 12:28:27     INFO -  1600259307965	Marionette	TRACE	Received observer notification message-manager-disconnect
[task 2020-09-16T12:28:27.976Z] 12:28:27     INFO -  1600259307965	Marionette	DEBUG	40 <- [1,26,null,["74"]]
[task 2020-09-16T12:28:27.978Z] 12:28:27     INFO -  1600259307966	Marionette	DEBUG	40 -> [0,27,"WebDriver:SwitchToWindow",{"handle":"74","focus":true}]
[task 2020-09-16T12:28:27.980Z] 12:28:27     INFO -  1600259307966	Marionette	DEBUG	40 <- [1,27,null,{"value":null}]
[task 2020-09-16T12:28:27.981Z] 12:28:27     INFO -  1600259307967	Marionette	DEBUG	40 -> [0,28,"WebDriver:GetChromeWindowHandles",{}]
[task 2020-09-16T12:28:27.983Z] 12:28:27     INFO -  1600259307967	Marionette	DEBUG	40 <- [1,28,null,["12"]]
[task 2020-09-16T12:28:27.985Z] 12:28:27     INFO -  1600259307967	Marionette	DEBUG	40 -> [0,29,"WebDriver:GetWindowHandles",{}]
[task 2020-09-16T12:28:27.987Z] 12:28:27     INFO -  1600259307967	Marionette	DEBUG	40 <- [1,29,null,["74"]]
[task 2020-09-16T12:28:27.989Z] 12:28:27     INFO -  1600259307968	Marionette	DEBUG	40 -> [0,30,"WebDriver:DeleteSession",{}]
[task 2020-09-16T12:28:27.990Z] 12:28:27     INFO -  1600259307969	Marionette	DEBUG	40 <- [1,30,null,{"value":null}]
[task 2020-09-16T12:28:27.992Z] 12:28:27     INFO -  1600259307970	Marionette	DEBUG	Closed connection 40
[task 2020-09-16T12:28:27.994Z] 12:28:27     INFO -  1600259307971	Marionette	DEBUG	Accepted connection 41 from 127.0.0.1:39878
[task 2020-09-16T12:28:27.997Z] 12:28:27     INFO -  1600259307971	Marionette	DEBUG	41 -> [0,1,"WebDriver:NewSession",{"strictFileInteractability":true}]
[task 2020-09-16T12:28:27.999Z] 12:28:27     INFO -  1600259307973	Marionette	TRACE	[74] Frame script loaded
[task 2020-09-16T12:28:28.001Z] 12:28:28     INFO -  1600259307981	Marionette	DEBUG	41 <- [1,1,null,{"sessionId":"36a64d19-09de-40c2-a911-7d3d8805b474","capabilities":{"browserName":"firefox","browserVersion":"82.0" ... mp/tmpcMDz2Q.mozrunner","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
[task 2020-09-16T12:28:28.003Z] 12:28:28     INFO -  1600259307985	Marionette	DEBUG	41 -> [0,2,"WebDriver:SetTimeouts",{"script":30000}]
[task 2020-09-16T12:28:28.004Z] 12:28:28     INFO -  1600259307986	Marionette	DEBUG	41 <- [1,2,null,{"value":null}]
[task 2020-09-16T12:28:28.006Z] 12:28:28     INFO -  1600259307988	Marionette	DEBUG	41 -> [0,3,"WebDriver:SetTimeouts",{"pageLoad":300000}]
[task 2020-09-16T12:28:28.010Z] 12:28:28     INFO -  1600259307989	Marionette	DEBUG	41 <- [1,3,null,{"value":null}]
[task 2020-09-16T12:28:28.012Z] 12:28:28     INFO -  1600259307991	Marionette	DEBUG	41 -> [0,4,"WebDriver:SetTimeouts",{"implicit":0}]
[task 2020-09-16T12:28:28.013Z] 12:28:28     INFO -  1600259307992	Marionette	DEBUG	41 <- [1,4,null,{"value":null}]
[task 2020-09-16T12:28:28.015Z] 12:28:28     INFO -  1600259307994	Marionette	DEBUG	41 -> [0,5,"WebDriver:DeleteSession",{}]
[task 2020-09-16T12:28:28.016Z] 12:28:28     INFO -  1600259307996	Marionette	DEBUG	41 <- [1,5,null,{"value":null}]
[task 2020-09-16T12:28:28.017Z] 12:28:28     INFO -  1600259307998	Marionette	DEBUG	Closed connection 41
[task 2020-09-16T12:28:28.018Z] 12:28:28     INFO -  1600259308001	Marionette	DEBUG	Accepted connection 42 from 127.0.0.1:39880
[task 2020-09-16T12:28:28.018Z] 12:28:28     INFO -  1600259308004	Marionette	DEBUG	42 -> [0,1,"WebDriver:NewSession",{"acceptInsecureCerts":true}]
[task 2020-09-16T12:28:28.019Z] 12:28:28     INFO -  1600259308006	Marionette	WARN	TLS certificate errors will be ignored for this session
[task 2020-09-16T12:28:28.020Z] 12:28:28     INFO -  1600259308008	Marionette	TRACE	[74] Frame script loaded
[task 2020-09-16T12:28:28.020Z] 12:28:28     INFO -  1600259308012	Marionette	DEBUG	42 <- [1,1,null,{"sessionId":"bf9103b4-f89b-4089-a30b-07a99dbf0ebf","capabilities":{"browserName":"firefox","browserVersion":"82.0" ... mp/tmpcMDz2Q.mozrunner","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
[task 2020-09-16T12:28:28.021Z] 12:28:28     INFO -  with safe session
[task 2020-09-16T12:28:28.029Z] 12:28:28     INFO -  1600259308022	Marionette	DEBUG	Closed connection 42
[task 2020-09-16T12:28:28.030Z] 12:28:28     INFO -  1600259308022	Marionette	DEBUG	Accepted connection 43 from 127.0.0.1:39882
[task 2020-09-16T12:28:28.031Z] 12:28:28     INFO -  1600259308023	Marionette	DEBUG	43 -> [0,1,"WebDriver:NewSession",{"acceptInsecureCerts":false}]
[task 2020-09-16T12:28:28.031Z] 12:28:28     INFO -  1600259308024	Marionette	TRACE	[74] Frame script loaded
[task 2020-09-16T12:28:28.032Z] 12:28:28     INFO -  1600259308025	Marionette	DEBUG	43 <- [1,1,null,{"sessionId":"ee715e84-59fa-4590-875b-8e46008d2a85","capabilities":{"browserName":"firefox","browserVersion":"82.0" ... mp/tmpcMDz2Q.mozrunner","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
[task 2020-09-16T12:28:28.033Z] 12:28:28     INFO -  1600259308025	Marionette	DEBUG	43 -> [0,2,"WebDriver:Navigate",{"url":"https://127.0.0.1:40182/test.html"}]
[task 2020-09-16T12:28:28.051Z] 12:28:28     INFO -  1600259308037	Marionette	TRACE	Received message beforeunload for http://127.0.0.1:43065/test.html
[task 2020-09-16T12:28:28.060Z] 12:28:28     INFO -  1600259308050	Marionette	TRACE	Received message beforeunload for http://127.0.0.1:43065/test.html
[task 2020-09-16T12:28:28.061Z] 12:28:28     INFO -  1600259308054	Marionette	TRACE	Received message pagehide for http://127.0.0.1:43065/test.html
[task 2020-09-16T12:28:28.096Z] 12:28:28     INFO -  with unsafe session
[task 2020-09-16T12:28:28.096Z] 12:28:28     INFO -  1600259308089	Marionette	TRACE	Received message DOMContentLoaded for about:certerror?e=nssBadCert&u=https%3A//127.0.0.1%3A40182/test.html&c=UTF-8&d=%20
[task 2020-09-16T12:28:28.098Z] 12:28:28     INFO -  1600259308090	Marionette	DEBUG	43 <- [1,2,{"error":"insecure certificate","message":"","stacktrace":"WebDriverError@chrome://marionette/content/error.js:181:5\nIn ... adyState@chrome://marionette/content/navigate.js:57:24\nonNavigation@chrome://marionette/content/navigate.js:302:43\n"},null]
[task 2020-09-16T12:28:28.100Z] 12:28:28     INFO -  1600259308090	Marionette	DEBUG	43 -> [0,3,"WebDriver:DeleteSession",{}]
[task 2020-09-16T12:28:28.101Z] 12:28:28     INFO -  1600259308091	Marionette	DEBUG	43 <- [1,3,null,{"value":null}]
[task 2020-09-16T12:28:28.103Z] 12:28:28     INFO -  1600259308092	Marionette	DEBUG	Closed connection 43
[task 2020-09-16T12:28:28.103Z] 12:28:28     INFO -  1600259308093	Marionette	DEBUG	Accepted connection 44 from 127.0.0.1:39894
[task 2020-09-16T12:28:28.103Z] 12:28:28     INFO -  1600259308093	Marionette	DEBUG	44 -> [0,1,"WebDriver:NewSession",{"acceptInsecureCerts":true}]
[task 2020-09-16T12:28:28.104Z] 12:28:28     INFO -  1600259308093	Marionette	WARN	TLS certificate errors will be ignored for this session
[task 2020-09-16T12:28:28.117Z] 12:28:28     INFO -  1600259308102	Marionette	TRACE	[74] Frame script loaded
[task 2020-09-16T12:28:28.118Z] 12:28:28     INFO -  1600259308106	Marionette	DEBUG	44 <- [1,1,null,{"sessionId":"a362f687-009f-413e-af18-6caafe2d473e","capabilities":{"browserName":"firefox","browserVersion":"82.0" ... mp/tmpcMDz2Q.mozrunner","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
[task 2020-09-16T12:28:28.118Z] 12:28:28     INFO -  1600259308107	Marionette	DEBUG	44 -> [0,2,"WebDriver:Navigate",{"url":"https://127.0.0.1:40182/test.html"}]
[task 2020-09-16T12:28:28.133Z] 12:28:28     INFO -  1600259308119	Marionette	TRACE	Received message beforeunload for about:certerror?e=nssBadCert&u=https%3A//127.0.0.1%3A40182/test.html&c=UTF-8&d=%20
[task 2020-09-16T12:28:28.141Z] 12:28:28     INFO -  1600259308133	Marionette	TRACE	Received message pagehide for about:certerror?e=nssBadCert&u=https%3A//127.0.0.1%3A40182/test.html&c=UTF-8&d=%20
[task 2020-09-16T12:28:28.149Z] 12:28:28     INFO -  1600259308144	Marionette	TRACE	Received message DOMContentLoaded for https://127.0.0.1:40182/test.html
[task 2020-09-16T12:28:28.149Z] 12:28:28     INFO -  1600259308144	Marionette	TRACE	Received message pageshow for https://127.0.0.1:40182/test.html
[task 2020-09-16T12:28:28.149Z] 12:28:28     INFO -  1600259308145	Marionette	DEBUG	44 <- [1,2,null,{"value":null}]
[task 2020-09-16T12:28:28.156Z] 12:28:28     INFO -  1600259308152	Marionette	DEBUG	44 -> [0,3,"WebDriver:DeleteSession",{}]
[task 2020-09-16T12:28:28.157Z] 12:28:28     INFO -  1600259308154	Marionette	DEBUG	44 <- [1,3,null,{"value":null}]
[task 2020-09-16T12:28:28.157Z] 12:28:28     INFO -  DEBUG: Adding blocker Transaction (1) for phase protections.sqlite#0: waiting for clients
[task 2020-09-16T12:28:28.157Z] 12:28:28     INFO -  DEBUG: Completed blocker Transaction (1) for phase protections.sqlite#0: waiting for clients
[task 2020-09-16T12:28:28.160Z] 12:28:28     INFO -  with safe session again
[task 2020-09-16T12:28:28.160Z] 12:28:28     INFO -  1600259308159	Marionette	DEBUG	Accepted connection 45 from 127.0.0.1:39900
[task 2020-09-16T12:28:28.161Z] 12:28:28     INFO -  1600259308160	Marionette	DEBUG	Closed connection 44
[task 2020-09-16T12:28:28.168Z] 12:28:28     INFO -  1600259308161	Marionette	DEBUG	45 -> [0,1,"WebDriver:NewSession",{"acceptInsecureCerts":false}]
[task 2020-09-16T12:28:28.172Z] 12:28:28     INFO -  1600259308170	Marionette	TRACE	[74] Frame script loaded
[task 2020-09-16T12:28:28.173Z] 12:28:28     INFO -  1600259308171	Marionette	DEBUG	45 <- [1,1,null,{"sessionId":"d5a695e8-924c-47b8-969a-aced9dc5b7d7","capabilities":{"browserName":"firefox","browserVersion":"82.0" ... mp/tmpcMDz2Q.mozrunner","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
[task 2020-09-16T12:28:28.174Z] 12:28:28     INFO -  1600259308172	Marionette	DEBUG	45 -> [0,2,"WebDriver:Navigate",{"url":"https://127.0.0.1:40182/test.html"}]
[task 2020-09-16T12:28:28.181Z] 12:28:28     INFO -  1600259308176	Marionette	TRACE	Received message beforeunload for https://127.0.0.1:40182/test.html
[task 2020-09-16T12:28:28.188Z] 12:28:28     INFO -  1600259308185	Marionette	TRACE	Received message pagehide for https://127.0.0.1:40182/test.html
[task 2020-09-16T12:28:28.196Z] 12:28:28     INFO -  1600259308194	Marionette	TRACE	Received message DOMContentLoaded for https://127.0.0.1:40182/test.html
[task 2020-09-16T12:28:28.196Z] 12:28:28     INFO -  1600259308194	Marionette	TRACE	Received message pageshow for https://127.0.0.1:40182/test.html
[task 2020-09-16T12:28:28.196Z] 12:28:28     INFO -  1600259308195	Marionette	DEBUG	45 <- [1,2,null,{"value":null}]
[task 2020-09-16T12:28:28.212Z] 12:28:28     INFO - TEST-UNEXPECTED-FAIL | testing/marionette/harness/marionette_harness/tests/unit/test_navigation.py TestTLSNavigation.test_deactivation | AssertionError: InsecureCertificateException not raised
[task 2020-09-16T12:28:28.212Z] 12:28:28     INFO - Traceback (most recent call last):
[task 2020-09-16T12:28:28.212Z] 12:28:28     INFO -   File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 196, in run
[task 2020-09-16T12:28:28.212Z] 12:28:28     INFO -     testMethod()
[task 2020-09-16T12:28:28.212Z] 12:28:28     INFO -   File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_navigation.py", line 767, in test_deactivation
[task 2020-09-16T12:28:28.212Z] 12:28:28     INFO -     session.navigate(invalid_cert_url)
[task 2020-09-16T12:28:28.212Z] 12:28:28     INFO - TEST-INFO took 235ms
[task 2020-09-16T12:28:28.212Z] 12:28:28     INFO -  1600259308202	Marionette	DEBUG	45 -> [0,3,"WebDriver:DeleteSession",{}]
[task 2020-09-16T12:28:28.212Z] 12:28:28     INFO -  1600259308203	Marionette	DEBUG	45 <- [1,3,null,{"value":null}]
[task 2020-09-16T12:28:28.215Z] 12:28:28     INFO -  1600259308213	Marionette	DEBUG	Closed connection 45
[task 2020-09-16T12:28:28.218Z] 12:28:28     INFO -  1600259308216	Marionette	DEBUG	Accepted connection 46 from 127.0.0.1:39906
[task 2020-09-16T12:28:28.219Z] 12:28:28     INFO -  1600259308217	Marionette	DEBUG	46 -> [0,1,"WebDriver:NewSession",{"strictFileInteractability":true}]
[task 2020-09-16T12:28:28.223Z] 12:28:28     INFO -  1600259308220	Marionette	TRACE	[74] Frame script loaded
[task 2020-09-16T12:28:28.230Z] 12:28:28     INFO -  1600259308227	Marionette	DEBUG	46 <- [1,1,null,{"sessionId":"a4aeadc9-d26c-49b1-810f-05ca9a872a48","capabilities":{"browserName":"firefox","browserVersion":"82.0" ... mp/tmpcMDz2Q.mozrunner","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
[task 2020-09-16T12:28:28.231Z] 12:28:28     INFO -  1600259308228	Marionette	DEBUG	46 -> [0,2,"WebDriver:DeleteSession",{}]
[task 2020-09-16T12:28:28.231Z] 12:28:28     INFO -  1600259308228	Marionette	DEBUG	46 <- [1,2,null,{"value":null}]
[task 2020-09-16T12:28:28.238Z] 12:28:28     INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_navigation.py TestTLSNavigation.test_navigate_by_click

Henrik, this seems to be caused by the changes in Bug 1493108, could you please take a look?

Flags: needinfo?(hskupin)

This is the same as bug 1582736 but looks like to perma-fail on beta on Linux only. I will see what I can do. In the worst case we just disable this intermittently failing test for that platform on beta.

Depends on: 1582736
Flags: needinfo?(hskupin)
OS: Unspecified → Linux
Hardware: Unspecified → All

Bug 1582736 should have fixed the failures. Can you please verify?

Severity: -- → S3
Flags: needinfo?(malexandru)
Priority: -- → P5
Assignee: nobody → mjzffr
Product: Testing → Remote Protocol
You need to log in before you can comment on or make changes to this bug.