Ralph Ferngold Reporter
	Description • 17 years ago

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
Build Identifier: Lightning 0.8pre build 2008010804 on Thunderbird 2.0.0.9 (20071031)

Lightning will corrupt ics files stored on my webdav server if I simply perform multiple updates quickly. It appears that lightning is attempting to update the ics file for each request asynchronously, thus causing the file to be truncated (of course I have no way of really knowing whats going on). After this happens, lightning is usually pretty hosed and won't refresh any calendars. Usually requires that I kill the thunderbird process (it doesn't die on exit) and restart after I have recovered my calendars from backup.


The webdav server is an apache 2.2 server running on linux. The URL being requested is https and uses basic authentication. 

Reproducible: Always

Steps to Reproduce:
1.Pick your favorite lightning view (I use the week view)
2.Make a number of changes to calendar events of your choice against a calendar set up as described in the but details.
3.

Actual Results:  
Within moments the ics will be be truncated and nothing further will work until thunderbird is killed and restarted.

Expected Results:  
All requested changes to the calendar should be executed successfully and reflected correctly in the ics file.

	Lars Wohlfahrt (thetux)
	Comment 1 • 17 years ago

Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.9) Gecko/20071031  Thunderbird/2.0.0.9 Lightning/0.8pre 2008010904


works for me

	Ralph Ferngold Reporter
	Comment 2 • 17 years ago

Additional information that might be useful: the ics file is about 100K in size the the connection is relatively slow (384kb). I believe this is important because the problem seems to be timing dependent: updates to the calendar that occur while the ics is being written. It happens far less frequently for smaller calendars and over faster connections.

	Steffen Sledz
	Comment 3 • 17 years ago

I can confirm this behaviour using a slow DSL connection (1024 kbit/s Down - und 128 kbit/s Up).

File size is 280kB. http://severinghaus.org/projects/icv/ has validated it positiv. 

	Lars Wohlfahrt (thetux)
	Comment 4 • 17 years ago

It's important to know: do you use any proxy servers?

	cmtalbert
	Comment 5 • 17 years ago

It sounds like from Comment 3, we can say that this is Confirmed.

I'm going to leave the QAWanted assessment in place, because it would be good to have a simple set of steps to test this with.  Steffen, can we use your webdav server? 

Thanks for the help ya'll.  It would also be good to check this with the latest 0.8 RC, which you can pick up here:
http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/nightly/0.8-candidates/rc1/

	Ralph Ferngold Reporter
	Comment 6 • 17 years ago

I tested the latest from the link above and the problem persists. Also, there are no proxy servers in any of the environments in which this problem occurs for me.

	Ralph Ferngold Reporter
	Comment 7 • 17 years ago

One other symptom I noticed: Not only does thunderbird not die after this occurs, but it pegs the CPU on one core of my box. Don't know if this is useful information.

	Gerhard Schneider
	Comment 8 • 17 years ago

I can confirm the bug on a 348k up DSL line - but only in 1 of 20-30 instances.
For me the truncated file size is always n*2048 Bytes. Hope it helps.

GS

	Steffen Sledz
	Comment 9 • 17 years ago

(In reply to comment #5)
> Steffen, can we use your webdav server?

Using my productive WebDAV server is not possible. Sorry. But I've checked with the WebDAV access to the Mediacenter of GMX.NET and have the same problem there. So this could be a good test server.

	Ralph Ferngold Reporter
	Comment 10 • 16 years ago

Any update on this bug? It's killing me.

	Gerhard Schneider
	Comment 11 • 16 years ago

As you can read in a letter to the editor in the 14/08 copy of the German c't magazine (p. 12) there is at least one other person having that problem (but only every 3 or 4 weeks).

	Markus Schumacher
	Comment 12 • 16 years ago

I have the same problem using Lightning0.8 (Build 2008033120) in TB 2.0.0.14 (20080421). 
It corrupts my calendar files almost every second update.  I'm using Mediacenter of GMX have a really slow internet connection but a rather big ics file (200kB).
It corrupts almost everytime a notification pops up, I'm really frustrated.

	Harald Böhme
	Comment 13 • 16 years ago

We have a similar setup here at my company (Apache 2.2 WebDAV)
This problem occurs also at LAN, while many user accessing the same ics file via WebDav. At some point the ics file get truncated.

	Daniel Boelzle [:dbo]
	Comment 14 • 16 years ago

I tried to reproduce the issue using GMX Media Center with a large calendar file without success w.r.t. file truncation. I get sporadic errors when stressing the calendar that PUT has failed, but a refresh reloads the file fine. This might be due to my good DSL connection, though.
Logging the locking and unlocking ics provider's action queue it looks good, i.e. it is kept locked until a PUT has been completed, thus before another PUT could happen.
What I spotted in the code is that there's a potential race when fetching the etag (GMX doesn't return it on PUT, so there's a follow-up PROPFIND). We don't wait for that until the next PUT is launched. However, this only might lead the etag check to fail, but is IMO no cause for file truncation.

(In reply to comment #13)
> We have a similar setup here at my company (Apache 2.2 WebDAV)
> This problem occurs also at LAN, while many user accessing the same ics file
> via WebDav. At some point the ics file get truncated.
I think this has been filed as bug 329570 resp. bug 290446. The latter is more like a solution proposal about whether to check etags or use locks.

	Philipp Kewisch [:Fallen] [:📆][:🧩]
	Comment 16 • 16 years ago

Comment on attachment 331296 [details] [diff] [review]
[checked in] fixing potential PROPFIND / unlock race

Not sure I like the fact that a callback function is used here, but I understand why you need to do so.

Since an alternative doesn't come to my mind,

r=philipp

	Daniel Boelzle [:dbo]
	Comment 17 • 16 years ago

Since I cannot reproduce the bug using e.g. MediaCenter with large ics files, could anyone try out a recent nightly whether the last patch makes it better?

	Ralph Ferngold Reporter
	Comment 18 • 16 years ago

Using the nightly build from Aug 05, the problem still occurs. I can cause it to happen with a few moments by simply moving an event around on the calendar a handful of times in quick succession. I also noticed a new behavior. On one occasion, lightning hung refreshing a calendar but the calendar was not truncated. Restarting thunderbird resolved that problem. 

	Daniel Boelzle [:dbo]
	Comment 19 • 16 years ago

I once again tried to reproduce the issue against GMX media center slowing down my DSL connection via ipfw pipes to modem/isdn speed without success.
Patches are really welcome, but unless we can safely reproduce the problem (which makes it almost impossible to fix), we won't block 0.9 on it.

	Ralph Ferngold Reporter
	Comment 20 • 16 years ago

I can set up access to my server if someone would like to test there. Send me an email and I'll send along connection info and credentials.

	Matthias R.
	Comment 21 • 16 years ago

My ics file get corrupted once a week. The calendar is also stored on the GMX media center. Could bug 390036 play a role?
@Daniel: Do you use IMAP/SMTP together with SSL?
I will disable SSL and report if this still happens. 

	Ralph Ferngold Reporter
	Comment 22 • 16 years ago

I switched all of my calendars from https to http. After about a week the problem has not recurred when normally it happens multiple times a week. It does sound like it may be related to bug 390036.  I had two occurrences of a calendar failing to reload after a modification. I had to kill the thunderbird process to get it to load, but the ics file was fine. No excessive CPU in this case.

	Daniel Boelzle [:dbo]
	Comment 23 • 16 years ago

(In reply to comment #21)
> @Daniel: Do you use IMAP/SMTP together with SSL?
Yes.

	Gerhard Schneider
	Comment 24 • 16 years ago

Another me, too.. (IMAP and SMTP with SSL)

	Bas van den Bosch
	Comment 25 • 15 years ago

I think this is fixed for 1.0pre by bug 329570 (concurrent editing) and the fixes in bug 390036 (hanging TB with Lightning with SSL). The latter contains a fix for 0.9 too (if you install the 0.9 fix in bug 363455).

	Daniel Boelzle [:dbo]
	Comment 26 • 15 years ago

Ralph, Steffen, Gerhard, Harald, Markus: Could you please check whether this problem still occurs using a recent lightning 1.0pre nightly? Thanks!

	Gerhard Schneider
	Comment 27 • 15 years ago

Not using TB 3.0pre and Lightning 1.0pre does not install on TB 2.0 - if I'm right..

	Philipp Kewisch [:Fallen] [:📆][:🧩]
	Comment 29 • 15 years ago

Does anyone experiencing this issue have apache server logs available? This might be a similar issue to bug 410874. This needs to be retested with Lightning 1.0b1 and Thunderbird 3.

	Philipp Kewisch [:Fallen] [:📆][:🧩]
	Comment 31 • 14 years ago

I'm going to close this bug for lack of testers. If this problem still occurs, please open a new bug.