Closed Bug 1070665 Opened 10 years ago Closed 5 years ago

TEST-UNEXPECTED-FAIL | /builds/slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_manual_attachment_reminder

Categories

(Thunderbird :: Message Compose Window, defect)

Product:
Thunderbird
Component:
Message Compose Window
Platform:
x86_64
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: iannbugzilla, Unassigned)

References

Details

(Keywords: intermittent-failure)

Attachments

(7 files)

part-01-startup.png: test starts within Xephyr virtual root window.
(deleted), image/png
Details
part-02-test-starts.PNG: C-C TB starts up and test begins.
(deleted), image/png
Details
part-03-test-proceeding.PNG: test chugs along with successful tests.
(deleted), image/png
Details
part-04-test-proceeding.png: test proceeds further to test "aggressive" handling.
(deleted), image/png
Details
part-05-000-failed.PNG: now comes the error. Send fails!
(deleted), image/png
Details
part-05-020-manully-moved-dialog.PNG: Displacing the modal dialog to explain the situation better.
(deleted), image/png
Details
part-05-025-failed-wider.PNG: To show the execution context.
(deleted), image/png
Details
TEST-START | /builds/slave/test/build/mozmill/composition/test-attachment-reminder.js | test_manual_attachment_reminder Step Pass: {"function": "Controller.keypress()"} Step Pass: {"function": "Controller.type()"} Step Pass: {"function": "Controller.type()"} Step Pass: {"function": "Controller.type()"} Step Pass: {"function": "controller.click()"} Step Pass: {"function": "Controller.keypress()"} Step Pass: {"function": "Controller.keypress()"} Step Pass: {"function": "Controller.keypress()"} Step Pass: {"function": "Controller.keypress()"} Step Pass: {"function": "controller.click()"} Step Pass: {"function": "Controller.keypress()"} Step Pass: {"function": "controller.click()"} Test Failure: Popup never opened! id=button-attachPopup, state=showing TEST-UNEXPECTED-FAIL | /builds/slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_manual_attachment_reminder
Potentially bug 1039452 needs reopening?
Depends on: 1039452
I will tackle this orange.
Inactive; closing (see bug 1180138).
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WORKSFORME
Still happens sometimes, e.g. https://treeherder.mozilla.org/#/jobs?repo=comm-central&revision=5106ef65c179 08:59:25 INFO - SUMMARY-UNEXPECTED-FAIL | test-attachment-reminder.js | test-attachment-reminder.js::test_manual_attachment_reminder 08:59:25 INFO - EXCEPTION: Timed out waiting for window open! 08:59:25 INFO - at: utils.js line 447 08:59:25 INFO - TimeoutError utils.js:447 13 08:59:25 INFO - waitFor utils.js:485 1 08:59:25 INFO - WindowWatcher_waitForWindowOpen test-window-helpers.js:247 1 08:59:25 INFO - wait_for_new_window test-window-helpers.js:577 1 08:59:25 INFO - wait_for_compose_window test-compose-helpers.js:218 21 08:59:25 INFO - test_manual_attachment_reminder test-attachment-reminder.js:332 9 08:59:25 INFO - Runner.prototype.wrapper frame.js:585 9 08:59:25 INFO - Runner.prototype._runTestModule frame.js:655 9 08:59:25 INFO - Runner.prototype.runTestModule frame.js:701 3 08:59:25 INFO - Runner.prototype.runTestDirectory frame.js:525 7 08:59:25 INFO - runTestDirectory frame.js:707 3 08:59:25 INFO - Bridge.prototype._execFunction server.js:179 10 08:59:25 INFO - Bridge.prototype.execFunction server.js:183 16 08:59:25 INFO - Session.prototype.receive server.js:283 3 08:59:25 INFO - AsyncRead.prototype.onDataAvailable server.js:88 3
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
The original report was probably about "popup never opened": SUMMARY-UNEXPECTED-FAIL | test-attachment-reminder.js | test-attachment-reminder.js::test_manual_attachment_reminder EXCEPTION: Popup never opened! id=button-attachPopup, state=showing at: utils.js line 447 TimeoutError utils.js:447 13 waitFor utils.js:485 1 _click_menus test-window-helpers.js:963 1 click_manual_reminder test-attachment-reminder.js:287 1 test_manual_attachment_reminder test-attachment-reminder.js:352 3 Runner.prototype.wrapper frame.js:585 9 Runner.prototype._runTestModule frame.js:655 9 Runner.prototype.runTestModule frame.js:701 3 Runner.prototype.runTestFile frame.js:534 3 runTestFile frame.js:713 3 Bridge.prototype._execFunction server.js:179 10 Bridge.prototype.execFunction server.js:183 16 Session.prototype.receive server.js:283 3 AsyncRead.prototype.onDataAvailable server.js:88 3 I can reproduce this one on my machine most of the time, but do not yet know what the problem is (apart from blaming it on the strange linux focus issues in mozmill)
There is an error for real. It happens all the time on my LOCAL test machine. But it looks slightly different from the one in comment 65 or 66. SUMMARY-UNEXPECTED-FAIL | test-attachment-reminder.js | test-attachment-reminder.js::test_disabling_attachment_reminder EXCEPTION: Timed out waiting for notification with value attachmentReminder to show. at: utils.js line 447 TimeoutError utils.js:447 13 waitFor utils.js:485 11 MozMillController.prototype.waitFor controller.js:687 3 wait_for_notification_to_show test-notificationbox-helpers.js:117 3 wait_for_reminder_state test-attachment-reminder.js:64 5 test_disabling_attachment_reminder test-attachment-reminder.js:623 3 Runner.prototype.wrapper frame.js:585 9 Runner.prototype._runTestModule frame.js:655 9 Runner.prototype.runTestModule frame.js:701 3 Runner.prototype.runTestFile frame.js:534 3 runTestFile frame.js:713 3 Bridge.prototype._execFunction server.js:179 10 Bridge.prototype.execFunction server.js:183 16 Session.prototype.receive server.js:283 3 AsyncRead.prototype.onDataAvailable server.js:88 3 SUMMARY-PASS | test-attachment-reminder.js::teardownModule /NREF-COMM-CENTRAL/comm-central/mozilla/../mail/testsuite-targets.mk:42: recipe for target 'mozmill-one' failed At least I think I know the reason for this failure. On my local test PC running linux, somehow fails to talk to (fake?) filelink server, and thus there is an error modal dialog popping up during execution. Unless this modal dialog is answered in time, mozmill kicks in due to time out, cancelling all other tests in /composite subdirectory. (This is bad: I just noticed that there is a strange failure NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIDOMWindowUtils.sendKeyEvent] causing some tests to fail on try-comm-central, but I have never seen this failure locally. Upon investigation, I realize a test, composition/test-charset-upgrade.js never gets executed on my local PC for the last 12 months or longer. The reason is composition/test-test-attachment-reminder.js caused the timeout and aborts all the following tests in /composition directory (including test-charset-upgrade.js). So this timeout issue is actually a grave test harness issue. (Even if the tests may succeed on other platforms, we may not exercise other tests thus cancelled very well under linux, possibly overlooking many failures caused by timing issues there as well.) Anyway, until today, I did not realize there are so many potential serious bugs lurking in the code. (I would think a test that experiences NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIDOMWindowUtils.sendKeyEvent] ought NOT to pass as a test.) Anyway, does anyone know how to modify the test code so that this will at least fail gracefully (and not cause timeout) so that mozmill can proceed with the other test files in /composition subdirectory? I will post the screen dump to explain how we can possibly handle this. BTW, I am using Xephyr for creating a local test execution environment aside from my usual X desktop. (I understand that mozmill seems to use a different virtual/remote desktop solution. But that should not cause any difference.) I have used X setup successfully for the last 3+ years. SIZE=1280x768 Xephyr -ac -br -noreset -screen $SIZE :2 & <--- This creates a virtual root desktop that <--- can be accessed via locahost:2.0 DISPLAY=localhost:2.0 waitfor 5 # oclock & xfwm4 & <----- This is the window manager for DISPLAY=localhost:2.0 Anyway, screen dump and the explanation of the error for the particular error mode I experience will follow. TIA
The single test file is excecuted by make mozmill-one SOLO_TEST=composition/test-attachment-reminder.js Immediately after the following, Xephyr window is empty. SIZE=1280x768 Xephyr -ac -br -noreset -screen $SIZE :2 & # <--- This creates a virtual root desktop that # <--- can be accessed via locahost:2.0 DISPLAY=localhost:2.0 waitfor 5 # oclock & xfwm4 & # <--- we need a desktop manager within it.
I takes a second or two before TB windows appears within Xephyr window and then test begins execution.
Testing proceed with successful tests. We can monitor the test progress within Xephyr window while we do useful things in the ordinary X desktop. They are virtually independent. (Using Different DISPLAY setup. One is localhost:2.0 and the other is at localhost:0.0) I wonder what other people do while running mozmill test. Presumably the virtual desktop solution used by others offer similar functionality.
Test proceeds further to the function test_attachment_reminder_aggressive_pref. As you can see, some tests do not close the windows used by test functions and thus we end up with many open windows. But I digress.
Now the error part. Modal error dialog with "Sending of the message faild" pops up, but this is not handled well. If we let it stay, eventually timeout occurs and mozmill kills the test then and there, and from what I see TB daemon for testing is kill, too. In the whole run of |make mozmill|, each subdirectory is executed one after the other, running the test files under the subdirectory in batch mode. Problem is, if there is a timeout, somehow the whole batch is aborted (because the TB daemon is killed?) and the subsequent test files in that subdirectory are not tested. So the test coverage suffers.
Beneath the modal dialog, there is something. I shifted the modal dialog to show what it is. It is a progress bar/monitor to show the status of sending of the message. This first appears, but as the sending fails, the modal error dialog appears. And this ought to be handled by |click_send_and_handle_send_error(cwc)|, but it does not seem to be handled at all.
In this very wide dump, I show the status of the progress of test code. The string "step-05" is in the code snippet below. So we are within click_send_and_handle_send_error() when the screen capture was done. NOTE THAT I CAN PROCEED BY HITTING [OK] dialog button manually to forcefully fail this test and let mozmill to proceed to run OTHER TESTS in /composite test subdirectory during the whole run of |make mozmill| !!! So my question is this. Does anyone know how I can let the test handle this condition? Do we have to modify click_send_and_handle_send_error(cwc) to handle this situation, or someone says, is there a mozmill focus issue that plagues this function, and this function is supposed to catch the error dialog as it is? Inquiring mind wants to know. Without this problem fixed, I don't get to see the execution of many files under /composition subdirectory locally (and presumably many others, too). This certainly hurts in terms of code coverage. (Not many people obviously read raw log dumps on try-comm-central regularly and notice these issues. Well, the timeout does not seem to occur on try-comm-central regularly during this particular test. So this is a moot issue as long as this particular test is concerned. But I see enough unexpected application close errors to suspect that some errors that may have been uncovered if the tests had been executed are simply not noticed since tests were cancelled due to the early timeout error. I mean there must have been a valid reason for timeout error on try-comm-central, and that reason might have affected OTHER tests, too. We simply don't know since the other tests are not executed.) Code with dumps. function test_attachment_vs_filelink_reminder() { // Open a blank message compose dump("step-01\n"); let cwc = open_compose_new_mail(); dump("step-01.5\n"); setup_msg_contents(cwc, "test@example.invalid", "Testing Filelink notification", "There is no body. I hope you don't mind!"); dump("step-02\n"); // There should be no notification yet. assert_any_notification(cwc, false); dump("step-03\n"); // Bring up the FileLink notification. let kOfferThreshold = "mail.compose.big_attachments.threshold_kb"; let maxSize = Services.prefs.getIntPref(kOfferThreshold, 0) * 1024; add_attachment(cwc, "http://www.example.com/1", maxSize); dump("step-04\n"); // The filelink attachment proposal should be up but not the attachment // reminder and it should also not interfere with the sending of the message. assert_notification_displayed(cwc, kBoxId, "bigAttachment", true); dump("step-04.5\n"); assert_automatic_reminder_state(cwc, false); cwc.sleep(2000); // add time to see the proper dialog. dump("step-05\n"); // we may see sending failed due to filelink not working (always on // local build test cases under linux. click_send_and_handle_send_error(cwc); <= we were here when the error appeared. dump("step-05.5\n"); close_window(cwc); } TIA
Interesting analysis. However, I can't see this problem locally. I'm debugging bug 1298081 that can be seen on trunk/try and that failure is in test_disabling_attachment_reminder . I can't see a problem in filelink test. Also, the report here was about a failure in test test_manual_attachment_reminder so I'm not sure your problem is related and should be in this bug. If you see compose windows left open from previous test, that should be a bug, those windows should be closed (unless the particural test failed). I've never seen 300 windows open as you say. So if you find a test that is not closing its windows when it has passed, please report it, we want those fixed.
And yes, click_send_and_handle_send_error() is supposed to click the "OK" in that error dialog. If it would accidentally get the progress dialog first, it should throw an error "Not a send error dialog; ..." If you didn't see that one in the log, there must be something more at play.
(In reply to :aceman from comment #75) > Interesting analysis. > However, I can't see this problem locally. > I'm debugging bug 1298081 that can be seen on trunk/try and that failure is > in test_disabling_attachment_reminder . I can't see a problem in filelink > test. After a few years of failures on one machine, and not on a different PC in a different place, I have come to the tentative conclusion that I am using a not so politically correct domain name example.com locally (firewalled, and separated from the Internet). This may interfere with the use of some test server name (domain names) used by C-C TB mozmill test. Bug 1299232 comment 2 - Enhance autoconfig MX lookup logic to check FOO.basedomain.tld on ISPDB in addition to basedomain.tld to support office365-hosted domains. See comment 2 there. > > Also, the report here was about a failure in test > test_manual_attachment_reminder so I'm not sure your problem is related and > should be in this bug. > Maybe I should separate my bug then. > If you see compose windows left open from previous test, that should be a > bug, those windows should be closed (unless the particural test failed). > I've never seen 300 windows open as you say. So if you find a test that is > not closing its windows when it has passed, please report it, we want those > fixed. That is Bug 960589 - Excessive number of windows created during |make mozmill| test run of thunderbird (comm-central) Actually, it is not a window left open from a previous test, but rather left open from a test function within a test file, etc. Still 300+ windows open (and window here counts the menu, etc. that is part of the toolkit and so the number of REAL TB windows is smaller, but still 300+ is a big number. My log from the weekend shows 362! (In reply to :aceman from comment #76) > And yes, click_send_and_handle_send_error() is supposed to click the "OK" in > that error dialog. If it would accidentally get the progress dialog first, > it should throw an error "Not a send error dialog; ..." If you didn't see > that one in the log, there must be something more at play. I don't see "Not a send error dialog; ..." So there must be something more about this issue. I will report back if I find something new, and that is probably I should create a new bugzilla. TIA

mozmill is gone.

Status: REOPENED → RESOLVED
Closed: 9 years ago → 5 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: