Closed
Bug 1070665
Opened 10 years ago
Closed 5 years ago
TEST-UNEXPECTED-FAIL | /builds/slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_manual_attachment_reminder
Categories
(Thunderbird :: Message Compose Window, defect)
Tracking
(Not tracked)
RESOLVED
WORKSFORME
People
(Reporter: iannbugzilla, Unassigned)
References
Details
(Keywords: intermittent-failure)
Attachments
(7 files)
TEST-START | /builds/slave/test/build/mozmill/composition/test-attachment-reminder.js | test_manual_attachment_reminder
Step Pass: {"function": "Controller.keypress()"}
Step Pass: {"function": "Controller.type()"}
Step Pass: {"function": "Controller.type()"}
Step Pass: {"function": "Controller.type()"}
Step Pass: {"function": "controller.click()"}
Step Pass: {"function": "Controller.keypress()"}
Step Pass: {"function": "Controller.keypress()"}
Step Pass: {"function": "Controller.keypress()"}
Step Pass: {"function": "Controller.keypress()"}
Step Pass: {"function": "controller.click()"}
Step Pass: {"function": "Controller.keypress()"}
Step Pass: {"function": "controller.click()"}
Test Failure: Popup never opened! id=button-attachPopup, state=showing
TEST-UNEXPECTED-FAIL | /builds/slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_manual_attachment_reminder
Potentially bug 1039452 needs reopening?
Depends on: 1039452
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 17•10 years ago
|
||
I will tackle this orange.
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 64•9 years ago
|
||
Inactive; closing (see bug 1180138).
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WORKSFORME
Comment 65•9 years ago
|
||
Still happens sometimes, e.g. https://treeherder.mozilla.org/#/jobs?repo=comm-central&revision=5106ef65c179
08:59:25 INFO - SUMMARY-UNEXPECTED-FAIL | test-attachment-reminder.js | test-attachment-reminder.js::test_manual_attachment_reminder
08:59:25 INFO - EXCEPTION: Timed out waiting for window open!
08:59:25 INFO - at: utils.js line 447
08:59:25 INFO - TimeoutError utils.js:447 13
08:59:25 INFO - waitFor utils.js:485 1
08:59:25 INFO - WindowWatcher_waitForWindowOpen test-window-helpers.js:247 1
08:59:25 INFO - wait_for_new_window test-window-helpers.js:577 1
08:59:25 INFO - wait_for_compose_window test-compose-helpers.js:218 21
08:59:25 INFO - test_manual_attachment_reminder test-attachment-reminder.js:332 9
08:59:25 INFO - Runner.prototype.wrapper frame.js:585 9
08:59:25 INFO - Runner.prototype._runTestModule frame.js:655 9
08:59:25 INFO - Runner.prototype.runTestModule frame.js:701 3
08:59:25 INFO - Runner.prototype.runTestDirectory frame.js:525 7
08:59:25 INFO - runTestDirectory frame.js:707 3
08:59:25 INFO - Bridge.prototype._execFunction server.js:179 10
08:59:25 INFO - Bridge.prototype.execFunction server.js:183 16
08:59:25 INFO - Session.prototype.receive server.js:283 3
08:59:25 INFO - AsyncRead.prototype.onDataAvailable server.js:88 3
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
Comment 66•9 years ago
|
||
The original report was probably about "popup never opened":
SUMMARY-UNEXPECTED-FAIL | test-attachment-reminder.js | test-attachment-reminder.js::test_manual_attachment_reminder
EXCEPTION: Popup never opened! id=button-attachPopup, state=showing
at: utils.js line 447
TimeoutError utils.js:447 13
waitFor utils.js:485 1
_click_menus test-window-helpers.js:963 1
click_manual_reminder test-attachment-reminder.js:287 1
test_manual_attachment_reminder test-attachment-reminder.js:352 3
Runner.prototype.wrapper frame.js:585 9
Runner.prototype._runTestModule frame.js:655 9
Runner.prototype.runTestModule frame.js:701 3
Runner.prototype.runTestFile frame.js:534 3
runTestFile frame.js:713 3
Bridge.prototype._execFunction server.js:179 10
Bridge.prototype.execFunction server.js:183 16
Session.prototype.receive server.js:283 3
AsyncRead.prototype.onDataAvailable server.js:88 3
I can reproduce this one on my machine most of the time, but do not yet know what the problem is (apart from blaming it on the strange linux focus issues in mozmill)
Comment 67•8 years ago
|
||
There is an error for real.
It happens all the time on my LOCAL test machine.
But it looks slightly different from the one in comment 65 or 66.
SUMMARY-UNEXPECTED-FAIL | test-attachment-reminder.js | test-attachment-reminder.js::test_disabling_attachment_reminder
EXCEPTION: Timed out waiting for notification with value attachmentReminder to show.
at: utils.js line 447
TimeoutError utils.js:447 13
waitFor utils.js:485 11
MozMillController.prototype.waitFor controller.js:687 3
wait_for_notification_to_show test-notificationbox-helpers.js:117 3
wait_for_reminder_state test-attachment-reminder.js:64 5
test_disabling_attachment_reminder test-attachment-reminder.js:623 3
Runner.prototype.wrapper frame.js:585 9
Runner.prototype._runTestModule frame.js:655 9
Runner.prototype.runTestModule frame.js:701 3
Runner.prototype.runTestFile frame.js:534 3
runTestFile frame.js:713 3
Bridge.prototype._execFunction server.js:179 10
Bridge.prototype.execFunction server.js:183 16
Session.prototype.receive server.js:283 3
AsyncRead.prototype.onDataAvailable server.js:88 3
SUMMARY-PASS | test-attachment-reminder.js::teardownModule
/NREF-COMM-CENTRAL/comm-central/mozilla/../mail/testsuite-targets.mk:42: recipe for target 'mozmill-one' failed
At least I think I know the reason for this failure.
On my local test PC running linux, somehow fails to talk to (fake?) filelink server, and
thus there is an error modal dialog popping up during execution.
Unless this modal dialog is answered in time, mozmill kicks in due to time out, cancelling all other tests
in /composite subdirectory.
(This is bad: I just noticed that there is a strange failure
NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIDOMWindowUtils.sendKeyEvent]
causing some tests to fail on
try-comm-central, but I have never seen this failure locally. Upon investigation, I realize
a test, composition/test-charset-upgrade.js never gets executed on my local PC for the last 12 months or longer.
The reason is composition/test-test-attachment-reminder.js
caused the timeout and aborts all the following tests in /composition directory (including test-charset-upgrade.js).
So this timeout issue is actually a grave test harness issue. (Even if the tests may succeed on other platforms, we may not exercise other tests thus cancelled very well under linux, possibly overlooking many failures caused by timing issues there as well.)
Anyway, until today, I did not realize there are so many potential serious bugs lurking in the code.
(I would think a test that experiences
NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIDOMWindowUtils.sendKeyEvent]
ought NOT to pass as a test.)
Anyway, does anyone know how to modify the test code so that this will at least fail gracefully
(and not cause timeout) so that mozmill can proceed with the other test files in /composition subdirectory? I will post the screen dump to explain how we can possibly handle this.
BTW, I am using Xephyr
for creating a local test execution environment aside from my usual X desktop.
(I understand that mozmill seems to use a different virtual/remote desktop solution. But that
should not cause any difference.) I have used X setup successfully for the last 3+ years.
SIZE=1280x768
Xephyr -ac -br -noreset -screen $SIZE :2 & <--- This creates a virtual root desktop that
<--- can be accessed via locahost:2.0
DISPLAY=localhost:2.0
waitfor 5
# oclock &
xfwm4 & <----- This is the window manager for DISPLAY=localhost:2.0
Anyway, screen dump and the explanation of the error for the particular error mode
I experience will follow.
TIA
Comment 68•8 years ago
|
||
The single test file is excecuted
by make mozmill-one SOLO_TEST=composition/test-attachment-reminder.js
Immediately after the following, Xephyr window is empty.
SIZE=1280x768
Xephyr -ac -br -noreset -screen $SIZE :2 &
# <--- This creates a virtual root desktop that
# <--- can be accessed via locahost:2.0
DISPLAY=localhost:2.0
waitfor 5
# oclock &
xfwm4 & # <--- we need a desktop manager within it.
Comment 69•8 years ago
|
||
I takes a second or two before TB windows appears within Xephyr window
and then test begins execution.
Comment 70•8 years ago
|
||
Testing proceed with successful tests.
We can monitor the test progress within Xephyr window while
we do useful things in the ordinary X desktop.
They are virtually independent. (Using Different DISPLAY setup.
One is localhost:2.0 and the other is at localhost:0.0)
I wonder what other people do while running mozmill test.
Presumably the virtual desktop solution used by others offer similar functionality.
Comment 71•8 years ago
|
||
Test proceeds further
to the function test_attachment_reminder_aggressive_pref.
As you can see, some tests do not close the windows used by test functions and thus we end up with many open windows. But I digress.
Comment 72•8 years ago
|
||
Now the error part.
Modal error dialog with "Sending of the message faild"
pops up, but this is not handled well.
If we let it stay, eventually timeout occurs and mozmill kills the test then and there, and from what I see
TB daemon for testing is kill, too.
In the whole run of |make mozmill|,
each subdirectory is executed one after the other, running the test files
under the subdirectory in batch mode.
Problem is, if there is a timeout, somehow the whole batch is aborted (because the TB daemon is killed?) and the subsequent test files in that subdirectory are not tested. So the test coverage suffers.
Comment 73•8 years ago
|
||
Beneath the modal dialog, there is something.
I shifted the modal dialog to show what it is.
It is a progress bar/monitor to show the status of sending of the message.
This first appears, but as the sending fails,
the modal error dialog appears.
And this ought to be handled by
|click_send_and_handle_send_error(cwc)|, but
it does not seem to be handled at all.
Comment 74•8 years ago
|
||
In this very wide dump, I show the status of the progress of test code.
The string "step-05" is in the code snippet below.
So we are within click_send_and_handle_send_error() when the screen capture was done.
NOTE THAT I CAN PROCEED BY HITTING [OK] dialog button manually
to forcefully fail this test and let mozmill to proceed to
run OTHER TESTS in /composite test subdirectory during the whole run of |make mozmill| !!!
So my question is this.
Does anyone know how I can let the test handle this condition?
Do we have to modify click_send_and_handle_send_error(cwc) to handle this situation, or
someone says, is there a mozmill focus issue that plagues this function, and
this function is supposed to catch the error dialog as it is?
Inquiring mind wants to know.
Without this problem fixed, I don't get to see the execution of many files
under /composition subdirectory locally (and presumably many others, too).
This certainly hurts in terms of code coverage.
(Not many people obviously read raw log dumps on try-comm-central regularly and notice these issues. Well, the timeout does not seem to occur on try-comm-central regularly during this particular test. So this is a moot issue as long as this particular test is concerned. But I see enough unexpected application close errors to suspect that some errors that may have been uncovered if the tests had been executed are simply not noticed since tests were cancelled due to the early timeout error. I mean there must have been a valid reason for timeout error on try-comm-central, and that reason might have affected OTHER tests, too. We simply don't know since the other tests are not executed.)
Code with dumps.
function test_attachment_vs_filelink_reminder() {
// Open a blank message compose
dump("step-01\n");
let cwc = open_compose_new_mail();
dump("step-01.5\n");
setup_msg_contents(cwc, "test@example.invalid", "Testing Filelink notification",
"There is no body. I hope you don't mind!");
dump("step-02\n");
// There should be no notification yet.
assert_any_notification(cwc, false);
dump("step-03\n");
// Bring up the FileLink notification.
let kOfferThreshold = "mail.compose.big_attachments.threshold_kb";
let maxSize = Services.prefs.getIntPref(kOfferThreshold, 0) * 1024;
add_attachment(cwc, "http://www.example.com/1", maxSize);
dump("step-04\n");
// The filelink attachment proposal should be up but not the attachment
// reminder and it should also not interfere with the sending of the message.
assert_notification_displayed(cwc, kBoxId, "bigAttachment", true);
dump("step-04.5\n");
assert_automatic_reminder_state(cwc, false);
cwc.sleep(2000); // add time to see the proper dialog.
dump("step-05\n");
// we may see sending failed due to filelink not working (always on
// local build test cases under linux.
click_send_and_handle_send_error(cwc); <= we were here when the error appeared.
dump("step-05.5\n");
close_window(cwc);
}
TIA
Comment 75•8 years ago
|
||
Interesting analysis.
However, I can't see this problem locally.
I'm debugging bug 1298081 that can be seen on trunk/try and that failure is in test_disabling_attachment_reminder . I can't see a problem in filelink test.
Also, the report here was about a failure in test test_manual_attachment_reminder so I'm not sure your problem is related and should be in this bug.
If you see compose windows left open from previous test, that should be a bug, those windows should be closed (unless the particural test failed). I've never seen 300 windows open as you say. So if you find a test that is not closing its windows when it has passed, please report it, we want those fixed.
Comment 76•8 years ago
|
||
And yes, click_send_and_handle_send_error() is supposed to click the "OK" in that error dialog. If it would accidentally get the progress dialog first, it should throw an error "Not a send error dialog; ..." If you didn't see that one in the log, there must be something more at play.
Comment 77•8 years ago
|
||
(In reply to :aceman from comment #75)
> Interesting analysis.
> However, I can't see this problem locally.
> I'm debugging bug 1298081 that can be seen on trunk/try and that failure is
> in test_disabling_attachment_reminder . I can't see a problem in filelink
> test.
After a few years of failures on one machine, and not on a different PC in a different place,
I have come to the tentative conclusion that I am using a not so politically correct domain name
example.com locally (firewalled, and separated from the Internet).
This may interfere with the use of some test server name (domain names) used by C-C TB mozmill test.
Bug 1299232 comment 2 - Enhance autoconfig MX lookup logic to check FOO.basedomain.tld on ISPDB in addition to basedomain.tld to support office365-hosted domains. See comment 2 there.
>
> Also, the report here was about a failure in test
> test_manual_attachment_reminder so I'm not sure your problem is related and
> should be in this bug.
>
Maybe I should separate my bug then.
> If you see compose windows left open from previous test, that should be a
> bug, those windows should be closed (unless the particural test failed).
> I've never seen 300 windows open as you say. So if you find a test that is
> not closing its windows when it has passed, please report it, we want those
> fixed.
That is Bug 960589 - Excessive number of windows created during |make mozmill| test run of thunderbird (comm-central)
Actually, it is not a window left open from a previous test, but rather
left open from a test function within a test file, etc.
Still 300+ windows open (and window here counts the menu, etc. that is part of the toolkit and so
the number of REAL TB windows is smaller, but still 300+ is a big number. My log from the weekend shows
362!
(In reply to :aceman from comment #76)
> And yes, click_send_and_handle_send_error() is supposed to click the "OK" in
> that error dialog. If it would accidentally get the progress dialog first,
> it should throw an error "Not a send error dialog; ..." If you didn't see
> that one in the log, there must be something more at play.
I don't see "Not a send error dialog; ..."
So there must be something more about this issue.
I will report back if I find something new, and that is probably I should create a new bugzilla.
TIA
Comment 78•5 years ago
|
||
mozmill is gone.
Status: REOPENED → RESOLVED
Closed: 9 years ago → 5 years ago
Resolution: --- → WORKSFORME
You need to log in
before you can comment on or make changes to this bug.
Description
•