Open Bug 1777035 Opened 2 years ago Updated 1 year ago

Intermittent toolkit/components/passwordmgr/test/mochitest/test_autocomplete_password_generation_confirm.html | single tracking bug

Categories

(Toolkit :: Password Manager, defect, P3)

Product:
Toolkit
Component:
Password Manager
Type:
defect

Tracking

()

People

(Reporter: jmaher, Assigned: joschmidt)

References

Details

(Keywords: intermittent-failure, intermittent-testcase, leave-open, Whiteboard: [stockwell disabled][fxcm-tech-debt][stockwell needswork:owner])

Attachments

(7 files)

Bug 1777035 - Disable test_autocomplete_password_generation_confirm.html on linux for frequent failures at least until a fix is in place. r=#intermittent-reviewers
(deleted), text/x-phabricator-request
Details
Bug 1777035 - fix intermittent test_autocomplete_password_generation_confirm.html - r?#credential-management-reviewers!
(deleted), text/x-phabricator-request
Details
Bug 1777035 - fix intermittent test_autocomplete_password_generation_confirm.html - r?#credential-management-reviewers!
(deleted), text/x-phabricator-request
Details
Bug 1777035 - flatten function calls - r?#credential-management-reviewers!
(deleted), text/x-phabricator-request
Details
Bug 1777035 - test in between password field - r?#credential-management-reviewers!
(deleted), text/x-phabricator-request
Details
Bug 1777035 - check all other password fields to be masked - r?#credential-management-reviewers!
(deleted), text/x-phabricator-request
Details
Bug 1777035 - re-enable test_autocomplete_password_generation_confirm.html on linux - r?#credential-management-reviewers!
(deleted), text/x-phabricator-request
Details
No description provided.

Additional information about this bug failures and frequency patterns can be found by running: ./mach test-info failure-report --bug 1777035

Severity: normal → S3

This bug has spiked in failures especially on linux asan since the patch in https://bugzilla.mozilla.org/show_bug.cgi?id=1833987#c23 landed.
A before/after failure rate can be seen here and here - https://treeherder.mozilla.org/intermittent-failures/bugdetails?startday=2023-06-01&endday=2023-06-08&tree=trunk&failurehash=all&bug=1777035
Johannes, please have a look over this one too. Thank you.

Flags: needinfo?(joschmidt)
Keywords: leave-open
Pushed by csabou@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/fa74019b770a Disable test_autocomplete_password_generation_confirm.html on linux for frequent failures at least until a fix is in place. r=credential-management-reviewers,sgalich
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]

Thank you for disabling! I have not managed to fix this yet.

Flags: needinfo?(joschmidt)
Assignee: nobody → joschmidt
Whiteboard: [stockwell disabled] → [stockwell disabled fxcm-tech-debt]
Duplicate of this bug: 1721900
Whiteboard: [stockwell disabled fxcm-tech-debt] → [stockwell disabled][fxcm-tech-debt]

There have been 83 total failures in the last 7 days.
There are:

  • 19 failures on macosx1015-64-qr debug and opt
  • 2 failures on macosx1015-64-shippable-qr opt
  • 6 failures on windows10-64-2009-qr opt
  • 7 failures on windows11-32-2009-qr opt and debug
  • 30 failures on windows11-64-2009-asan-qr opt
  • 19 failures on windows11-64-2009-qr debug

Recent failure log.

Johannes, is there any chance you could look into this?
It has an extremely high frequency on macOS and Win ( left after the linux disable) - and it is also present on release trees ( 55 failures in the last 7 days on esr115).

Should we disable this test completely? Do you have time to look into this?
Thank you.

Flags: needinfo?(joschmidt)

This is taking longer than expected. Can't give an ETA yet. If not ready until tonight, I'd propose we disable it completely for now

Flags: needinfo?(joschmidt)
Attachment #9343871 - Attachment description: WIP: - Bug 1777035 - fix intermittent test_autocomplete_password_generation_confirm.html → Bug 1777035 - fix intermittent test_autocomplete_password_generation_confirm.html - r?#credential-management-reviewers!
Pushed by joschmidt@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/7af2b04879fc fix intermittent test_autocomplete_password_generation_confirm.html - r=credential-management-reviewers,jneuberger

Hi Natalia, I hope this is a fix. I will monitor the failure rate and if it goes down, I'd re-enable the test on linux, too. Otherwise lets disable completely.

(In reply to Johannes from comment #82)

Hi Natalia, I hope this is a fix. I will monitor the failure rate and if it goes down, I'd re-enable the test on linux, too. Otherwise lets disable completely.

Thank you for taking the time to put up a patch!

Pushed by csabou@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/23e906719520 re-enable test_autocomplete_password_generation_confirm.html after a potential fix. a=test-only
Whiteboard: [stockwell disabled][fxcm-tech-debt] → [fxcm-tech-debt]

This still has a consistent failure rate on linux as seen here. Let's see how it goes on mac and windows but it might end up being disabled everywhere.
Backout link: https://hg.mozilla.org/integration/autoland/rev/b0f00c8e055766816efa02a825aa5c1f7bd11be9

Whiteboard: [fxcm-tech-debt] → [stockwell disabled][fxcm-tech-debt]

On windows the failure rate looks something like this. On retriggers in failed 8 times out of 11.

I'm working on a new fix

Attachment #9344242 - Attachment description: WIP: - Bug 1777035 - fix intermittent test_autocomplete_password_generation_confirm.html → Bug 1777035 - fix intermittent test_autocomplete_password_generation_confirm.html - r?#credential-management-reviewers!

Depends on D183772

Depends on D183795

Update

There have been 80 total failures within the last 7 days:

  • 26 failures on OS X 10.15 WebRender opt/debug
  • 2 failures on OS X 10.15 WebRender Shippable opt
  • 6 failures on windows10-64-2009-qr debug
  • 5 failures on Windows 11 x86 22H2 WebRender debug
  • 32 failures on Windows 11 x64 22H2 asan WebRender opt
  • 1 failure on Windows 11 x64 22H2 CCov WebRender opt
  • 8 failures on Windows 11 x64 22H2 WebRender debug

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=424090605&repo=autoland&lineNumber=13254

Whiteboard: [stockwell disabled][fxcm-tech-debt] → [stockwell disabled][fxcm-tech-debt][stockwell needswork:owner]

There have been 91 total failures in the last 7 days.
There are:

  • 24 failures on macosx1015-64-qr opt and debug
  • 4 failures on macosx1015-64-shippable-qr opt
  • 3 failures on windows10-64-2009-qr debug
  • 10 failures on windows11-32-2009-qr debug
  • 36 failures on windows11-64-2009-asan-qr opt
  • 14 failures on windows11-64-2009-qr debug

Recent failure log.

Johannes, do you have any updates? Thank you

Flags: needinfo?(joschmidt)

Hi, my patch is still in review

Flags: needinfo?(joschmidt)

Hi Johannes! Your patch is accepted. Can you please land it, maybe we can get rid of these failures.
Thank you!

Flags: needinfo?(joschmidt)

There have been 60 total failures in the last 7 days.
There are:

  • 12 failures on macosx1015-64-qr debug
  • 2 failures on macosx1015-64-shippable-qr opt
  • 2 failures on windows10-64-2009-qr opt
  • 8 failures on windows11-32-2009-qr opt
  • 27 failures on windows11-64-2009-asan-qr opt
  • 9 failures on windows11-64-2009-qr debug

Hi Johannes! Your patch is accepted. Can you please land it? Thank you!

Any chance to get the patch landed soon to resolve the frequent failures?

Next thing on my agenda!

Flags: needinfo?(joschmidt)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: