Closed Bug 273369 Opened 20 years ago Closed 17 years ago

Can't download emails with large attachment

Categories

(Thunderbird :: General, defect)

Product:
Thunderbird
Component:
General
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: h-bauer, Assigned: mscott)

References

Details

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0
Build Identifier: Thunderbird v1.0RC1-german, also v0.9-english

Trying to download emails with a large attachment (>3MB) thunderbird seems to
download (running progress bar, incoming network traffic) but does not show a
new email in the incoming folder. Downloading only the header works perfect.
Next trying to download the content using the link in the main section of the
mail again results in the above behaviour (progress bar, traffic) but does not
show any downloaded or updated content. Emails without or with a small
attachment are downloaded without any problem! The affected pop-accounts are at
'gmx.de' and 'compuserve.de'.

Curiously the same problems happen to me with pegasus 4.21c since i use
WinXP-Sp2. I use the global inbox which is taken to a user defined directory.
Mozilla mail does not show the above mentioned download problems. Deactivating
my firewall (of WinXP-Sp2) and my antivir software (H+BEDV Antivir) did not
change anything. Compressing the incoming folder does not enlarge to see any
additional mail (https://bugzilla.mozilla.org/show_bug.cgi?id=261668). So it
seems to be another bug (maybe familiar).

The above mentioned download problem is happening every time i try to download
an email with a large attachment. Other users told me that they only had the
problem as long as they have the antivir software running. Some other users
don't have the problem at all. But there are others sharing the same problem
(it's not imaginary). You may look to the german discussion at
http://www.thunderbird-mail.de/forum/viewtopic.php?t=5342 &
http://www.thunderbird-mail.de/forum/viewtopic.php?t=4941.

It might be hard to reproduce the error. Maybe comparing the code with mozilla
mail could help. If i can do anything to help you reproducing the error, please
let me know.

Thanks for a great work
Hans Bauer

Reproducible: Always
Steps to Reproduce:
1. Download emails with large attachment from a pop-account
2.
3.

Actual Results:  
See above: Download traffic but no additional email received


I'm connecting to the internet through a wlan-router and a dsl modem.
Sorry, but i forgot to say that i did not restrict the size of the emails to be
downloaded (checkbox to save space in the account-settings is not checked).
Get POP3 protocol log when "mail scan by client-side anti-virus software is
inactive", with no download size limitation, not "Fetch header only".
See http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#pop for
protocol log.
Please note that log file is overlayed on restart.

To confirm that "mail scan is inactive", see Server Settings of Account setting
and confirm that server name is real POP3 server and login-id is real login-id
before test.

If problem does not occur when "mail scan by client-side anti-virus software is
inactive", get protocol log with "mail scan active".
To confirm that "mail scan active", see Server Settings of Account setting and
confirm that server name is local mail proxy of anti-virus software and login-id
is login id to ant-virus software.
See manual of anti-virus software for servername/login-id of local mail proxy by
anti-virus software.
Do not paste log data here.
Attach log file through "Create a New Attachment".
Before attach log, remove your login-id data from log for safety, since
authentification is not a problem in this bug's case.

Another question.
Are there enough space for TMP directry?
Go MS DOS Prompt and enter "Set", then see TEMP and TMP settings, and see
directry listing of these two directries.
(Usually TEMP under user's "ducument setting" directry, then C: drive") 
Dear m-wada,

thank you for your hints. I report what i have done:
1. My "TEMP" and "TMP" both link to "c:\documents...\user\locale...\Temp" which
is write protected with a size of 181MB. My c-drive has a free size of 37.3GB.
The dir command from this directory shows 40054575104 Bytes free.

2. I confirmed in the server settings of my gmx-account:
- Server-type: POP Mail-Server
- Server: "pop.gmx.net" with Port "110"
- Only "Automatically download new mails" is checked
- All other checkboxes are unchecked (No fetch headers only, no download size
limitation)
and deactivated my H-BEDV-Antivir Software. I did not find a checkbox for "mail
scan by client side antivirus software" in the settings of my gmx-account.

3. According to your link to
"http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#pop" i created a
file "tb.bat" for capturing the thunderbird log containing the 3 lines
- set NSPR_LOG_MODULES=protocol:5
- set NSPR_LOG_FILE=d:\temp\tb.log
- start c:\programme\thunderbird\thunderbird.exe

4. Calling "tb.bat" thunderbird was opened and i clicked to download a mail with
an attachment >3MB from my gmx-account. It seems that thunderbird downloads but
no new mail was shown (as described above). A file "tb.log" was created and i
looked into this file: It was empty when thunderbird was opened from the batch
"tb.bat". It remains empty as long as the download was processed and it remains
empty after i closed thunderbird.

Anything i did wrong?
Thanks a lot.
Hans Bauer
Maybe the bug is initiated already before the download starts:

When i click to get new mail from my gmx-account thunderbird shows in the
statusline "Connected with pop.gmx.net" and stops. No running progress bar is
shown and nothing else happens. I have to click the button once more to start
the download (running progress bar) and get incoming network traffic. After the
download no new mail is shown as described above.

Hope this could help
Thanks
Hans Bauer
You have to change the batch file according to this one:

set NSPR_LOG_MODULES=POP3:5
set NSPR_LOG_FILE=d:\temp\tb.log
start c:\programme\thunderbird\thunderbird.exe

Please try it again. Can you empty your inbox before and compress it so no other
mail is inside? Please attach the msf file after you downloaded the whole mail.
It seems that there is no reference to that mail inside the msf file.

This bug seems to be related or identical to bug 239943.
Thank you a lot for your corrections.

I did what you asked for, emptied and compressed the inbox-folder and finally
tried a new download. You may now see the log-file and the msf-file created
during and after the download at

http://www.h-bauer.de/TB/tb_upload_041207.html

Looking at the logfile it seems that there is a gmx-timeout although i had a
continuously incoming traffic during download. Is gmx waiting for a missed
feedback from thunderbird during the download?
(In reply to comment #7)
Your log says ;
0[274800]: RECV: +OK mailbox has 1 messages (5171634 octets)
(snip)
0[274800]: SEND: STAT
(snip)
0[274800]: RECV: +OK 1 5171634
(snip)
0[274800]: SEND: LIST
0[274800]: RECV: +OK mailbox has 1 messages (5171634 octets)
(snip)
0[274800]: RECV: 1 5171634
(snip)
0[274800]: SEND: UIDL
(snip)
0[274800]: RECV: +OK mailbox has 1 messages (5171634 octets)
(snip)
0[274800]: RECV: 1 f9a1c51b1feb55407c2d1571c4ed1786
(snip)
0[274800]: SEND: RETR 1

0[274800]: OnResponseTimeout: username=xxxxxxx
0[274800]: OnResponseTimeout: username=xxxxxxx

No data was sent from server to Mozilla even though server said data length is
5171634 octets.
So Mozilla wait for data, then time out occured.  
This is completely same situation as Bug 253886.

A suspect, most suspectable suspect, is anti-virus software on mail server.
Next secenario is possible(This is my conclusion on Bug 253886).
  (1) Mail server save mail data length(and other info) in control file,
      then save a mail in a file.
      (Some mail server save a mail in a file, eg. Qmail, and other mails a file)
  (2) Anti-virus software deletes the mail file or remove data from the file.
  (3) Mail server sends mail data length in control file, 5171634 octets,
      to Mozilla.
  (4) Mail server starts to send mail data to Moziiia.
      But file is deleted.
      So server believes all data has been sent when EOF(End of file),
      and ends process, then no data is sent to client.
  (5) Mozilla waits for data since server said he will send 5171634 octets.
      Then wait for data, and time out occurs.

(2) is completely same situation as Bug 116443, although victim is different,
victim of your problem is mail server and victim of Bug 116443 is Mozilla. 

Hans Bauer, can you contact to gmx.de(or real mail server, if GMX is only a
proxy) and ask whether above guess is right or not? 

Another possibility.
Since GMX says "Proxy", cause can be mail size limitaion of proxy server,
if the proxy sends back mail data to RETR request after all mail data is
received form mail server. 
Dear m-wada,

Sorry, but i can't follow your arguments. If, as you say, the mailserver is the
victim for this problem, why is the download performed without any problem using
mozilla mail? The problem to discuss here only occurs using thunderbird or
pegasus 4.21c. My settings for gmx are the same for thunderbird, mozilla mail
and pegasus.

Thanks a lot for your efforts
Hans Bauer
(In reply to comment #10)
> why is the download performed without any problem using mozilla mail?
Do you mean that the mail of the protocol log of Comment #7 was successufully
down loaded by Mozilla? 
Just information.
Bug 270826 reported that crash occured when large(3MB+) attachment.
Thunderbird only problem when mail size is larger than 2MB or 4MB?
Hans Bauer, you say no problem if "Fetch header only" even when large attachment.
Can you test additional case?
What happens when download size is limited for the mail of comment #7?
 (1) When 4MB, (2) When 2MB, (3) When 1MB

Dear m-wada,

emails with a small or without attachments are downloaded without any problem
using thunderbird. Downloading emails with an attachment >3MB can be done using
mozilla mail using the same account with the same account settings. Your are
right, the download of the file according to #7 is performed without any problem
using mozilla mail. To enable you seeing the difference (thunderbird - mozilla
mail), i created a log-file for mozilla mail that can be seen at

http://www.h-bauer.de/TB/mm_upload_041207.html

I hope this might help to find the bug.
Thank you all a lot
Hans Bauer
Another Thunderbird bug when large mail.
Bug 226707 : Impossible to download big mail (4,1MB)
Dear m-wada,

fetching the headers is not a problem with thunderbird for all kind of emails
with no, small or large attachments. The headers of all mails are downloaded
correctly. The download of the content using the link in the main section of the
mail can only be performed for emails with no or small attachments. Trying to
download the content of an email with a large attachment will result incoming
traffic but will not show any updated content (timeout as above?).

Hope this helps
Thanks
Hans Bauer
I can't believe "Bug 226707" is the same. For my problem thunderbird is not
locked at any time and i never got a message 'disk full or no permission to
write file'.
(In reply to comment #17)

WORKSFORME for next mails with Thunderbird /nightly/latest-0.9 2004/12/01 build 
(Win-2K).
(Mail 1)
  Subject:     Large Mail 1
  Body-text  = Large Mail 1
  Attachment-1 =  705 KB(PDF file)
  Attachment-2 = 1794 KB(PDF file)
  Total Mail size after receieve = 3423 KB 
(Mail 2)
  Subject:     Large Mail 2
  Body-text  = Large Mail 2
  Attachment = 7874 KB(thunderbird-win32.zip)
  Total Mail size after receieve = 10790 KB 
Thunderbird received above mails successufully.
# Large mail is not a problem since account on SpyMac, free 1GB mail storage :-)

What is the difference between your case and my case?

> I can't believe "Bug 226707" is the same.
Sorry, this is absolutely different bug.
Dear m-wada,

Sorry, but i cannot understand what you want with #17:
- Asking what's the difference (difference of what) or
- Saying that it's an absolutely different bug
Thanks
Hans Bauer
(In reply to comment #19)

Sorry,  "Inreply to comment #17" is for "absolutely different bug".

WORKSFORME and "what's the difference" is for this bug's problem.
(Mail 2) in my comment #18 was successfully downloaded by my Thunderbird.
But your mail of your comment #7 could not be downloaded by your Thunderbird.
My question is - why result is different.
Dear m-wada,

the problem is to reproduce the error, i know. I tried to explain this in #1 by
giving the hint to compare the download code of thunderbird and mozilla mail.
Another way to find the bug might be to compare the logfiles of mozilla mail
(works perfect, see #14) and thunderbird (fails, see #7). I know that my problem
only occurs for some people. Other people can solve the problem by deactivating
the antivir-software but some others share my problem.

I'd be happy if i could give you more hints. So for any questions, dont hesitate
to contact me.
Thanks
Hans Bauer
my theory is that the client is timing out on the really large message fetches.
Perhaps you're running a version of Mozilla that doesn't have the client timeout
code, but a version of thunderbird that does. You can shutdown tbird, go find
your prefs.js in your user profile directory and and change the timeout by
adding the following line:

user_pref("mail.pop3_response_timeout", 200);

That will change the timout to over 3 minutes, which should be sufficient. If
you need more help in finding and editing prefs.js, let me know.


Same Problem! I can't download E-Mails with an attachment bigger than 300kb.
TB starts to download them, but then it stops without downloading any email!

This problem does not exist with version 0.8, in 0.9 it first appered and is
still in 1.0!
Markus, can you attach a pop3 protocol log? See comments 3 and 4 for instructions.
It seems to me, that TB tries to download the message and the maybe the
conection to the server is cancled, because when I try to download the message
again I get an error message "No conection posible, is there still another
session active?"

After about 10 mins I can try to download the message again, but again nothing
happens (#23)

(In reply to comment #22)

> 
> user_pref("mail.pop3_response_timeout", 200);
> 
 
This works fine for me! Thanks!

I think, that I've you've got a slow conection to the internet and/or a slow
email server, the conection may time out...? Just an idea...! Hope my english is
not too bad :-)

Hi David,

you might have a look to my thunderbird and my mozilla mail log. Both are
created using the same gmx-account, the same settings and the same email to be
downloaded (bmp-attachment with 3.7MB). The logs are provided at
http://www.h-bauer.de/TB/tb_upload_041207.html
and
http://www.h-bauer.de/TB/mm_upload_041207.html
The mozilla mail log documents a successful download, the thunderbird  was not
finished completely (i believe) to show a new mail in the in-folder.

Hopefully this might help.
Thank you all
Hans Bauer
we probably need a UI for adjusting the timeout - maybe like the slider thing
that OE uses.
Please David, no timeout slider.

There should not be a timeout as long as there is an incoming traffic. The
restricion is already provided with the limit for the download size you can give
in the settings. Providing a slider for timeout will not prevent from questions
and hotline requests as given in this thread. Thunderbird should act the same
way (intuitive) as mozilla mail.

Don't you think?
Yours Hans Bauer
I don't agree, if I'm understanding this problem. Our timeout code will not
trigger if there's any incoming traffic. We start the timer when we send a
command, and we kill it the second we get any data back for that command, and we
don't start the timer again until we've finished processing the data and we're
waiting for more data. From the log I saw, we issued a RETR command, and no data
at all came back within the timeout period. It looks like for large messages,
your server takes a very long time to even start sending the data for the message.
Sorry to say David, this is wrong.

The status dialog for the network connection shows a lot of incoming traffic as
soon as thunderbird begins do download.  There is about 5 MB of incoming traffic
wihthin about 40 seconds before the timeout occurs. After the timeout the
connection is closed and all the traffic is lost.
but the protocol log doesn't show that - is there a virus checker or something
processing the data before giving any of it to thunderbird?
(In reply to comment #31)
> Sorry to say David, this is wrong.
> 
> The status dialog for the network connection shows a lot of incoming traffic as
> soon as thunderbird begins do download.  There is about 5 MB of incoming traffic
> wihthin about 40 seconds before the timeout occurs. After the timeout the
> connection is closed and all the traffic is lost.

I see absoluly the same!
Hi David

As i told in my first message, i deactivated all antivir and firewall software
and got the same behavour. I connect cia a router and dsl to the internet. I
don't know any intermediate software filtering any traffic on my pc
(In reply to comment #32)
> but the protocol log doesn't show that - is there a virus checker or something
> processing the data before giving any of it to thunderbird?

I think this is possible! On an other Computer wich has no firewall and
antivirus software, the problem did not occure, but shouldn't there be more
people with this problem? I'm using the XP SP2 Firewall + F Secure Internet
Security 2004 wich also icludes a Firewall and a Virus scanner!
Hi all,

Yes there are people able to avoid the problem by disabling antivir software (i
mentioned this before), but there are also people disabling antivir and the
problem consists. Another point to consider: The attached file was a bitmap file
(*.bmp) and should not be filtered by any software.
Thanks Hans Bauer

I can't confirm this bug with version 1.0 (20041206) under win2k. The mail
(approach 4MB in size) was downloaded successfully within 4mins over a slow
internet connection. I tested with my account by gmx.de. I havn't enabled any
Firewall. Perhaps I will try that later.
Dear all,

today i dowloaded thunderbird v1.0-english and installed it over my
v1.0RC1-german. Nothing changed. Afterwards i completely deinstalled
thunderbird. I also deleted the folder 'c:\documents...\user\...\Thunderbird' to
remove all my account settings. Then i restarted my pc (WinXP-Prof.-Sp2),
confirmed that the installation directory was deleted, installed thunderbird
1.0-english as new and received the same problem dowloading mails with
attachments >3MB.

To simplify usage of this thread i try to summarize the experiences:
====================================================================
My Problem:
Thunderbird v0.9-english, v.1.0RC1-german and v1.0-english only seems to dowload
emails with large attachments (here a bmp-file >3MB) but does not show these
emails in the inbox. Downloading only the header works fine, but clicking on the
link in the main section of the mail to download the complete content fails as
before (incoming traffic, no new mail in the inbox). Emails without or with
small attachments are downloaded and shown correctly by thunderbird. This
happens for my pop-accounts at 'gmx.de', 'compuserve.de', 'strato.de',
'web.de',.... Yet the problem cannot be reproduced from other people in a
reliable way.

My installation:
a. I'm working with WinXP-Prof.-Sp2 (german) and connect to the internet via
wlan-router and dsl-modem.
b. Thunderbird is configured to download all mails from my accounts without
size-restiction (see thunderbird::account settings::disk-space::To save disk
space... is unchecked).
c. I use the global inbox.
d. There is enough free space (>37GB) in my TMP-/TEMP-directory (in the Dos-Box
enter 'set' to find) .

Found out and questions:
1. The same problem for emails with large attachments happen to me using pegasus
v4.21c since i work with my new pc with WinXP-Prof.-Sp2 installed. On my old pc
with Win2K-Sp4 (destroyed) i never had such problems with pegasus.

2. The problem doesn't occur using mozilla mail v.1.7.2 (english) on the new pc
also used for thunderbird. Mozilla mail is able to download the affected email
from the same gmx-account with the same settings and the same network
connection. The log-file for mozilla shows receiving traffic and can be seen at
http://www.h-bauer.de/TB/mm_upload_041207.html.

3. The log-file of thunderbird documents a timeout and shows no receiving
traffic for thunderbird, but the status window for the network connection shows
incoming traffic as long as thunderbird seems to download (running progress
bar). The logfile can be seen at http://www.h-bauer.de/TB/tb_upload_041207.html.

4. The file 'inbox.msf' does not show the email with the bmp-attachment >3MB
after trying the download. The content of 'inbox.msf' can be seen at
http://www.h-bauer.de/TB/tb_upload_041207.html.

5. Deactivating my H+BEDV-Antivir-software and WinXP-Sp2-firewall did not change
anything for me. I don't know any additional software on my pc that could
prevent the incoming traffic from reaching thunderbird. I also don't know why
any intermediate software should prevent the mail to be delivered since the
attachment of the affected email is a bmp-file without the risk of viruses or
worms (Note: emails with smaller attachments are delivered fine). But some
people told, deactivating their firewall helped.

6. I sent the affected email to the gmx-account of 'Henrik Skupin'. He could
download the mail with thunderbird without any problem. So the encoding of the
bmp-attachment by pegasus (i use pegasus to send mail) should not be buggy.

7. When i click to get new mail from my gmx-account, thunderbird shows in the
statusline "Connected with pop.gmx.net" and stops. No running progress bar is
shown and nothing else happens. I have to click the button once more to start
the download (running progress bar) and get incoming network traffic for my pc
(maybe not reaching thunderbird). Might this initiate the bug?

Does anyone have an idea what's going wrong with thunderbird on my pc? Any help
will be appreciated.
Thank you all for your patience, efforts and help!
Yours
Hans Bauer
Hi all,

i believe i found the victim for the timeout by preventing thunderbird to
receive emails with attachments >3MB.

Only deactivating H+BEDV-Antivir on the running machine did not cause any
changes. If i deactivate H+BEDV-Antivir, disable all related autostarts and
restart my pc, thunderbird will download all emails (with no, small or large
attachments) without any problem. There is also no more necessity to press the
download button twice. Maybe this is a problem caused by H-BEDV-Antivir in
conjunction with WinXP-Prof-Sp2, because with Win2K H+BEDV-Antivir did not cause
the problems before.

I will contact H+BEDV-Antivir and hold you up to date. Sorry for all inconveniences.
Thank you all for your help and assistance.
Hans Bauer
Hans, did you try setting the pref? I understand you don't think it will work,
but I can't tell if you tried it...

user_pref("mail.pop3_response_timeout", 200);
Dear David,

i believe, changing the timeout might be a workaround but no bugfix. I tried to
locate the bug to prevent users from falling into the same hole. So i confess, i
did not try increasing the timeout.

I hope you can understand this. You may read my #39 at bugzille, i believe i
found the victim of the problem and will try to force the developers to solve
the bug.

I really want to thank you for your hints.
Yours Hans Bauer
Dear David,

According to your request to test, if only increasing the timeout will enable
thunderbird to download emails with larger attachments even for my configuration, i
- reactivated H+BEDV-Antivir,
- restarted my pc,
- wrote 'user_pref("mail.pop3_response_timeout", 400);' into my 'prefs.js'
- and tried to download the email with an attachment >3MB.

The download was sucessful. So increasing the timeout may be used as workaround
for this bug caused by H+BEDV-Antivir (maybe in conjunction with WinXP-Sp2). I
hope this helps that thunderbird is not blamed by this problem and could offer a
solution, even if the solution is not satisfactory (for me).

Once more: Thank you all for your great work and your exemplary assistance.
Yours, allways willing to help you if i can, don't hesitate to ask. 
Hans Bauer
Hi all,

the malfunction is now confirmed by H+BEDV Antivir. The H+BEDV development team
is working to solve the problem. As soon as there are new informations, i will
post here.

For your interest, next follows the link to my appropriate thread in the german
thunderbird forum:

http://www.thunderbird-mail.de/forum/viewtopic.php?p=31730#31730

Thank you all
Hans Bauer
For me the download of mails with attachement > 3 mb works. 

But there was no progressbar in tb. I could see that tb loads a lot because the
network traffic was increasing. (it wasn't my default account)
Set Bug 127461 to "Depends on:" of this bug, because mail.pop3_response_timeout
was introduced by Bug 127461.
Depends on: 127461
I've opened Bug 277071.
Bug 277071 : Timeout by mail.pop3_response_timeout should be notified to user
Comfirming according to protocol log.
Status: UNCONFIRMED → NEW
Ever confirmed: true
So this is now WFM for everyone?
QA Contact: general
JFY.
mail.pop3_response_timeout had been obsoleted and new mailnews.tcptimeout (deafult=60sec.) was already introduced.
If 60(sec.) is small for your environment, try larger timeout value of mailnews.tcptimeout.

No response and no new comment posting means "problem varnished" in many cases.
Closing as WORKSFORME.
Status: NEW → RESOLVED
Closed: 17 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.