Filed by: rgurzau [at] mozilla.com https://treeherder.mozilla.org/logviewer.html#?job_id=190427405&repo=autoland https://queue.taskcluster.net/v1/task/MkMiIMdBQyycSnaWtCko6Q/runs/0/artifacts/public/logs/live_backing.log 00:30:28 INFO - 283 INFO TEST-START | browser/components/payments/test/browser/browser_card_edit.js 00:30:29 INFO - TEST-INFO | started process screenshot 00:30:29 INFO - TEST-INFO | screenshot: exit 0 00:30:29 INFO - Buffered messages logged at 00:30:28 00:30:29 INFO - 284 INFO Entering test bound setup_head 00:30:29 INFO - 285 INFO Leaving test bound setup_head 00:30:29 INFO - 286 INFO Entering test bound test_add_link 00:30:29 INFO - 287 INFO waitForMessageFromWidget: paymentDialogReady 00:30:29 INFO - 288 INFO Got paymentDialogReady from widget 00:30:29 INFO - 289 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Got payment widget - 00:30:29 INFO - 290 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | requestId should be defined - 00:30:29 INFO - 291 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | dialog should not be closed - 00:30:29 INFO - 292 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Got payment frame - 00:30:29 INFO - 293 INFO dialog ready 00:30:29 INFO - 294 INFO helper functions injected into frame 00:30:29 INFO - 295 INFO Console message: [JavaScript Error: "Content Security Policy: The page’s settings blocked the loading of a resource at inline (“default-src”)." {file: "resource://payments/paymentRequest.xhtml" line: 0}] 00:30:29 INFO - 296 INFO verifyPersistCheckbox 00:30:29 INFO - 297 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | checkbox should be visible when adding a new address - true == true - 00:30:29 INFO - 298 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | persist checkbox is in the expected state - true == true - 00:30:29 INFO - Buffered messages logged at 00:30:29 00:30:29 INFO - 299 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Add title should be set - "Add Credit Card" == "Add Credit Card" - 00:30:29 INFO - 300 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | isPrivate flag has expected value when shown from a private/non-private session - false == false - 00:30:29 INFO - 301 INFO fillInCardForm: fill the form with card: {"cc-exp-month":12,"cc-exp-year":2027,"cc-name":"Jane McMaster-Card","cc-number":"5555555555554444"} 00:30:29 INFO - 302 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field #cc-exp-month shouldn't be disabled - true == true - 00:30:29 INFO - 303 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | 12 - "12" == true - 00:30:29 INFO - 304 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field #cc-exp-year shouldn't be disabled - true == true - 00:30:29 INFO - 305 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | 2027 - "2027" == true - 00:30:29 INFO - 306 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field #cc-name shouldn't be disabled - true == true - 00:30:29 INFO - 307 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Jane McMaster-Card - "Jane McMaster-Card" == true - 00:30:29 INFO - 308 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field #cc-number shouldn't be disabled - true == true - 00:30:29 INFO - 309 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | 5555555555554444 - "5555555555554444" == true - 00:30:29 INFO - 310 INFO ,2018,2019,2020,2021,2022,2023,2024,2025,2026,2027,2028 00:30:29 INFO - 311 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | The billing address selector should always be visible - true == true - 00:30:29 INFO - 312 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Only 2 child options should exist by default - 2 == 2 - 00:30:29 INFO - 313 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | The first option should be the blank/empty option - "" == "" - 00:30:29 INFO - 314 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | "a47c1d57247c" == true - 00:30:29 INFO - 315 INFO navigateToAddAddressPage: check were on the expected page first 00:30:29 INFO - 316 INFO current page state: basic-card-page waiting for: basic-card-page 00:30:29 INFO - 317 INFO navigateToAddAddressPage: click the link 00:30:29 INFO - 318 INFO navigateToAddAddressPage: wait for address page 00:30:29 INFO - 319 INFO Console message: [JavaScript Error: "Content Security Policy: The page’s settings blocked the loading of a resource at inline (“default-src”)." {file: "resource://payments/paymentRequest.xhtml" line: 0}] 00:30:29 INFO - 320 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Address on add address page should be correct - "Add Billing Address" == "Add Billing Address" - 00:30:29 INFO - 321 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Add title should be still be on credit card page - "Add Credit Card" == "Add Credit Card" - 00:30:29 INFO - 322 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field should still have previous value entered - "12" == 12 - 00:30:29 INFO - 323 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Fields should still be enabled for editing - true == true - 00:30:29 INFO - 324 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field should still have previous value entered - "2027" == 2027 - 00:30:29 INFO - 325 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Fields should still be enabled for editing - true == true - 00:30:29 INFO - 326 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field should still have previous value entered - "Jane McMaster-Card" == "Jane McMaster-Card" - 00:30:29 INFO - 327 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Fields should still be enabled for editing - true == true - 00:30:29 INFO - 328 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field should still have previous value entered - "5555555555554444" == "5555555555554444" - 00:30:29 INFO - 329 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Fields should still be enabled for editing - true == true - 00:30:29 INFO - 330 INFO navigateToAddAddressPage: check were on the expected page first 00:30:29 INFO - 331 INFO current page state: basic-card-page waiting for: basic-card-page 00:30:29 INFO - 332 INFO navigateToAddAddressPage: click the link 00:30:29 INFO - 333 INFO navigateToAddAddressPage: wait for address page 00:30:29 INFO - 334 INFO Console message: [JavaScript Error: "Content Security Policy: The page’s settings blocked the loading of a resource at inline (“default-src”)." {file: "resource://payments/paymentRequest.xhtml" line: 0}] 00:30:29 INFO - 335 INFO manuallyAddAddress: fill the form with address: {"given-name":"Timothy","additional-name":"Johann","family-name":"Berners-Lee","organization":"World Wide Web Consortium","street-address":"1 Pommes Frittes Place","address-level2":"Berlin","address-level1":"BE","postal-code":"02138","country":"DE","tel":"+16172535702","email":"timbl@example.org"} 00:30:29 INFO - 336 INFO fillInAddressForm, persistCheckbox.checked: true 00:30:29 INFO - 337 INFO verifyPersistCheckbox 00:30:29 INFO - 338 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | checkbox should be visible when adding a new address - true == true - 00:30:29 INFO - 339 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | persist checkbox is in the expected state - true == true - 00:30:29 INFO - 340 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Primary button should not be disabled when clicking it - true == true - 00:30:29 INFO - 341 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Check address was added - 2 == 2 - 00:30:29 INFO - 342 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | preserveFieldValues should be set when coming back from address-page - true == true - 00:30:29 INFO - 343 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | billingAddressGUID should be set when coming back from address-page - "c4838cead4a1" == true - 00:30:29 INFO - 344 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Three options should exist in the billingAddressSelect - 3 == 3 - 00:30:29 INFO - 345 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | The select should have the new address selected - "c4838cead4a1" == "c4838cead4a1" - 00:30:29 INFO - 346 INFO fillInCardForm: fill the form with card: {"cc-exp-month":12,"cc-exp-year":2027,"cc-name":"Jane McMaster-Card","cc-number":"5555555555554444"} 00:30:29 INFO - 347 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field #cc-exp-month shouldn't be disabled - true == true - 00:30:29 INFO - 348 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | 12 - "12" == true - 00:30:29 INFO - 349 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field #cc-exp-year shouldn't be disabled - true == true - 00:30:29 INFO - Buffered messages finished 00:30:29 ERROR - 350 INFO TEST-UNEXPECTED-FAIL | browser/components/payments/test/browser/browser_card_edit.js | 2027 - "" == true - 00:30:29 INFO - Stack trace: 00:30:29 INFO - resource://testing-common/content-task.js line 59 > eval:null:19 00:30:29 INFO - 351 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field #cc-name shouldn't be disabled - true == true - 00:30:29 INFO - 352 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Jane McMaster-Card - "Jane McMaster-Card" == true - 00:30:29 INFO - 353 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field #cc-number shouldn't be disabled - true == true - 00:30:29 INFO - 354 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | 5555555555554444 - "5555555555554444" == true - 00:30:29 INFO - 355 INFO ,2018,2019,2020,2021,2022,2023,2024,2025,2026,2027,2028 00:30:29 INFO - 356 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Primary button should not be disabled when clicking it - true == true - 00:30:29 INFO - 357 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Card was added - 2 == 2 - 00:30:29 INFO - 358 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | No temporary cards addded - 0 == 0 - 00:30:29 INFO - 359 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | New card was saved - 2 == 2 - 00:30:29 INFO - 360 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | All cc-numbers in state are masked - true == true - 00:30:29 INFO - 361 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Check cc-exp-month - 12 == 12 - 00:30:29 INFO - Not taking screenshot here: see the one that was previously logged 00:30:29 ERROR - 362 INFO TEST-UNEXPECTED-FAIL | browser/components/payments/test/browser/browser_card_edit.js | Check cc-exp-year - "undefined" == 2027 - 00:30:29 INFO - Stack trace: 00:30:29 INFO - resource://testing-common/content-task.js line 59 > eval:null:36 00:30:29 INFO - 363 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Check cc-name - "Jane McMaster-Card" == "Jane McMaster-Card" - 00:30:29 INFO - 364 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Check cc-number - "************4444" == "************4444" - 00:30:29 INFO - 365 INFO TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | The saved card should be associated with the billing address - "c4838cead4a1" == "c4838cead4a1" - 00:30:29 INFO - 366 INFO acknowledging the completion from the merchant page 00:30:29 INFO - GECKO(3164) | JavaScript error: chrome://payments/content/paymentDialogWrapper.js, line 189: TypeError: cardData['cc-exp-year'] is undefined 00:30:29 INFO - 367 INFO Console message: [JavaScript Error: "TypeError: cardData['cc-exp-year'] is undefined" {file: "chrome://payments/content/paymentDialogWrapper.js" line: 189}] 00:30:29 INFO - Not taking screenshot here: see the one that was previously logged 00:30:29 ERROR - 368 INFO TEST-UNEXPECTED-FAIL | browser/components/payments/test/browser/browser_card_edit.js | [JavaScript Error: "TypeError: cardData['cc-exp-year'] is undefined" {file: "chrome://payments/content/paymentDialogWrapper.js" line: 189}] - 00:30:29 INFO - Stack trace: 00:30:29 INFO - chrome://mochitests/content/browser/browser/components/payments/test/browser/head.js:onConsoleMessage:329 00:30:29 INFO - chrome://mochikit/content/tests/SimpleTest/specialpowersAPI.js:observe/<:377 00:30:29 INFO - jar:file:///Z:/task_1532649843/build/application/firefox/browser/omni.ja!/components/paymentUIService.js:showPayment:49 00:35:21 INFO - 369 INFO Console message: [JavaScript Error: "getScreenshot(https://example.com/browser/browser/components/payments/test/browser/blank_page.html) failed: TypeError: NetworkError when attempting to fetch resource." {file: "resource://activity-stream/lib/Screenshots.jsm" line: 47}] 00:35:21 INFO - getScreenshotForURL@resource://activity-stream/lib/Screenshots.jsm:47:7 00:35:21 INFO - async*maybeCacheScreenshot@resource://activity-stream/lib/Screenshots.jsm:101:32 00:35:21 INFO - async*_fetchScreenshot@resource://activity-stream/lib/TopSitesFeed.jsm:231:11 00:35:21 INFO - async*_fetchIcon@resource://activity-stream/lib/TopSitesFeed.jsm:219:11 00:35:21 INFO - async*getLinksWithDefaults@resource://activity-stream/lib/TopSitesFeed.jsm:158:11 00:35:21 INFO - async*refresh@resource://activity-stream/lib/TopSitesFeed.jsm:180:25 00:35:21 INFO - async*onAction@resource://activity-stream/lib/TopSitesFeed.jsm:400:9 00:35:21 INFO - _middleware/</<@resource://activity-stream/lib/Store.jsm:51:11 00:35:21 INFO - Store/this[method]@resource://activity-stream/lib/Store.jsm:29:55 00:35:21 INFO - init/this.intervalId<@resource://activity-stream/lib/SystemTickFeed.jsm:18:41 00:35:21 INFO - notify@resource://gre/modules/Timer.jsm:42:7 00:35:21 INFO - showPayment@jar:file:///Z:/task_1532649843/build/application/firefox/browser/omni.ja!/components/paymentUIService.js:49:5 00:35:21 INFO - Not taking screenshot here: see the one that was previously logged

There are 40 failures in the last 7 days, 24 on Windows7-32, 8 on windows10-64 and 1 on windows7-32-msvc This file has been modified with bug 1476204, specifically https://hg.mozilla.org/integration/autoland/rev/c63340378a529cb5140d0b1db81988ba4d13a89d Matt, can you take a look?

After retriggers, I believe that the job started to fail on this push: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=45cf26c3c53c96eae955c24a1b4b7b05fb9ef6ea

Sorry, I was on PTO from Monday to Wednesday. I will try look at this today.

(In reply to Matthew N. [:MattN] (PM if requests are blocking you) from comment #8) > Sorry, I was on PTO from Monday to Wednesday. I will try look at this today. No problem, thank you.

The test is trying to set the cc-exp-year field to a particular value and then verify the field value. From the screenshot it looks like the focus isn't on the expected field, and the form is missing labels. So maybe we need a better way to wait for the form to be ready?

Volume seemed to go back down.

There is a total of 3 failures in the last 7 days, on windows7-32 pgo, windows10-64-msvc opt and windows10-64 opt. Recent failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=194593010&repo=autoland&lineNumber=3901 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Add link text - "Add" == "Add" - 19:15:43 INFO - Console message: [JavaScript Error: "Content Security Policy: The page’s settings blocked the loading of a resource at inline (“default-src”)." {file: "resource://payments/paymentRequest.xhtml" line: 0}] 19:15:43 INFO - fillInCardForm: fill the form with card: {"cc-exp-month":1,"cc-exp-year":2027,"cc-name":"John Doe","cc-number":"4111111111111111"} 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field #cc-exp-month shouldn't be disabled - true == true - 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | 01 - "1" == true - 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field #cc-exp-year shouldn't be disabled - true == true - 19:15:43 INFO - Buffered messages finished 19:15:43 INFO - TEST-UNEXPECTED-FAIL | browser/components/payments/test/browser/browser_card_edit.js | 2027 - "" == true - 19:15:43 INFO - Stack trace: 19:15:43 INFO - resource://testing-common/content-task.js line 59 > eval:null:19 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field #cc-name shouldn't be disabled - true == true - 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | John Doe - "John Doe" == true - 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Field #cc-number shouldn't be disabled - true == true - 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | 4111111111111111 - "4111111111111111" == true - 19:15:43 INFO - ,2018,2019,2020,2021,2022,2023,2024,2025,2026,2027,2028 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | No card was saved in state - 1 == 1 - 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Card was added temporarily - 1 == 1 - 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Check there is one card - 1 == 1 - 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Check cc-exp-month 1 matches 1 - "1" == 1 - 19:15:43 INFO - Not taking screenshot here: see the one that was previously logged 19:15:43 INFO - TEST-UNEXPECTED-FAIL | browser/components/payments/test/browser/browser_card_edit.js | Check cc-exp-year matches 2027 - "" == 2027 - 19:15:43 INFO - Stack trace: 19:15:43 INFO - resource://testing-common/content-task.js line 59 > eval:null:29 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | Check cc-name John Doe matches John Doe - "John Doe" == "John Doe" - 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | cc-number is masked - "************1111" == "************1111" - 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | cc-given-name was computed - "John" == "John" - 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | cc-family-name was computed - "Doe" == "Doe" - 19:15:43 INFO - Not taking screenshot here: see the one that was previously logged 19:15:43 INFO - TEST-UNEXPECTED-FAIL | browser/components/payments/test/browser/browser_card_edit.js | cc-exp was computed - "" == true - 19:15:43 INFO - Stack trace: 19:15:43 INFO - resource://testing-common/content-task.js line 59 > eval:null:35 19:15:43 INFO - TEST-PASS | browser/components/payments/test/browser/browser_card_edit.js | cc-number-encrypted was computed - "MEIEEPgAAAAAAAAAAAAAAAAAAAEwFAYIKoZIhvcNAwcECFJqBJhDyme6BBgPSLH/l29ZLUypBtuMW9Gn91+1lLpZr5E=" == true - 19:15:43 INFO - Leaving test bound test_private_card_adding 19:15:43 INFO - Console message: SENTINEL 19:15:43 INFO - GECKO(5788) | MEMORY STAT | vsize 699MB | vsizeMaxContiguous 541MB | residentFast 219MB | heapAllocated 83MB 19:15:43 INFO - TEST-OK | browser/components/payments/test/browser/browser_card_edit.js | took 16487ms 19:15:43 INFO - checking window state

Part of the issue here is that ok() is used instead of is(). Three arguments are passed to ok(), https://searchfox.org/mozilla-central/rev/07463fbd625219eea917f5b040d34b1627ab2ce7/browser/components/payments/test/browser/head.js#509

Comment on attachment 9010467 [details] Bug 1478880 - Wait for field to get focus before sending keys to it. r?MattN Matthew N. [:MattN] (PM me if requests are blocking you) has approved the revision.

I have a try push passing but I need to bisect it to figure out what actually fixed the problem: https://treeherder.mozilla.org/#/jobs?repo=try&revision=722cf206a82cd0e3d1293b95eb3fa52c3d5bd10e

Pushed by mozilla@noorenberghe.ca: https://hg.mozilla.org/integration/mozilla-inbound/rev/e7e27205f4cd payments/test/browser/head.js: fillInCardForm reliability fixes. r=MattN