Closed
Bug 361730
Opened 18 years ago
Closed 13 years ago
Seamonkey _deletes_ mail file if the mail file's size is greater than 2GiB and the .msf file has zero length
Categories
(MailNews Core :: Backend, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: baldauf--2015--bugzilla.mozilla.org, Assigned: Bienvenu)
Details
(Keywords: dataloss, fixed1.8.0.9, fixed1.8.1.1, Whiteboard: needs landing)
Attachments
(1 file, 1 obsolete file)
|
(deleted),
patch
|
wtc
:
review+
mscott
:
superreview+
dveditz
:
approval1.8.0.9+
|
Details | Diff | Splinter Review |
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.8) Gecko/20060911 SUSE/1.0.6-0.1 SeaMonkey/1.0.6
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.8) Gecko/20060911 SUSE/1.0.6-0.1 SeaMonkey/1.0.6
At a friends computer, I could reproduce severe data loss. It seems that Seamonkey Mail cannot handle files with length >2GiB, at least on i686 (32 bit machine), and goes wild :-(
I could verify this using strace:
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop/Inbox", {st_mode=S_IFREG|0644, st_size=596951668, ...}) = 0
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop/Trash", {st_mode=S_IFREG|0600, st_size=199594990, ...}) = 0
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop/Sent", {st_mode=S_IFREG|0600, st_size=2148532224, ...}) = 0
[pid 9402] open("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop/Sent", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666 <unfinished ...>
[pid 9404] <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
[pid 9404] gettimeofday({1164334227, 629206}, NULL) = 0
[pid 9404] futex(0x80a5368, FUTEX_WAKE, 1) = 0
[pid 9404] gettimeofday({1164334227, 629358}, NULL) = 0
[pid 9404] gettimeofday({1164334227, 629414}, NULL) = 0
[pid 9404] clock_gettime(CLOCK_REALTIME, {1164334227, 629469000}) = 0
[pid 9404] futex(0x80a53d0, FUTEX_WAIT, 251, {1, 187945000} <unfinished ...>
[pid 9402] <... open resumed> ) = 47
[pid 9402] lseek(47, 0, SEEK_CUR) = 0
[pid 9402] lseek(47, 0, SEEK_END) = 0
[pid 9402] lseek(47, 0, SEEK_SET) = 0
[pid 9402] close(47) = 0
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop/Drafts", {st_mode=S_IFREG|0600, st_size=50566887, ...}) = 0
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop/Templates", {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
[pid 9402] open("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 47
[pid 9402] fstat64(47, {st_mode=S_IFDIR|0755, st_size=744, ...}) = 0
[pid 9402] fcntl64(47, F_SETFD, FD_CLOEXEC) = 0
[pid 9402] mmap2(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb1cbd000
[pid 9402] getdents64(47, /* 24 entries */, 131072) = 824
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop", {st_mode=S_IFDIR|0755, st_size=744, ...}) = 0
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop/Sent.msf", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop/Sent", {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop", {st_mode=S_IFDIR|0755, st_size=744, ...}) = 0
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop/Drafts.msf", {st_mode=S_IFREG|0644, st_size=3863, ...}) = 0
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop/Drafts", {st_mode=S_IFREG|0600, st_size=50566887, ...}) = 0
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop", {st_mode=S_IFDIR|0755, st_size=744, ...}) = 0
At first, seamonkey gets stat data for that file "Mail/pop/Sent". You see, the file has size 2148532224, as "st_size=2148532224" tells.
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop/Sent", {st_mode=S_IFREG|0600, st_size=2148532224, ...}) = 0
Then seamonkey opens the file for writing, using the O_TRUNC flag, which means that the file is to be truncated (its length set to zero)
[pid 9402] open("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop/Sent", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666 <unfinished ...>
We can even verify this result, as now the file size is 0, as "st_size=0" tells.
[pid 9402] stat64("/home/heike/.mozilla/heike/zqf8bswn.slt/Mail/pop/Sent", {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
Reproducible: Always
Steps to Reproduce:
1. Create a very large mail file in the appropriate directory. In my specific case, this was "Mail/pop/Sent" file. This file should be greater than 2GiB. Using "dd if=/dev/zero of=Sent bs=1M count=2049" is fine.
2. Create the corresponding .msf file (e.g. "Mail/pop/Sent,msf"). This file should have zero length. Using "rm Sent.msf", "touch Sent.msf" is fine.
3. Start Seamonkey Mail
4. Look at the size of the Mail file.
Actual Results:
The size of "Sent" is 0
Expected Results:
The size of "Sent" should remain greater than 2GiB.
This problem has been verified both on Seamonkey 1.0.2 and Seamonkey 1.0.6.
|
||
Comment 1•18 years ago
|
||
I can confirm this for SeaMonkey 1.1b [Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061105 SeaMonkey/1.1b].
Status: UNCONFIRMED → NEW
Ever confirmed: true
|
Reporter | |
Comment 2•18 years ago
|
||
So what is this http://lxr.mozilla.org/seamonkey/source/xpcom/io/nsLocalFileUnix.cpp#1110 ?
The source code line
LL_UI2L(*aFileSize, (PRUint32)mCachedStat.st_size);
resolves to
(*aFileSize) = (PRInt64) (PRUint32) mCachedStat.st_size;
This may not be the reason for this bug, but this is still silly! First down-convert to 32 bits and then up-convert to 64 bits again.
|
||
Comment 3•18 years ago
|
||
compare bug 240192
|
|
||
Comment 4•18 years ago
|
||
JFTR: SeaMonkey 1.1b [Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1) Gecko/20061102 SeaMonkey/1.1b] is *NOT* affected, it already seems to be using 64 bit stuff.
I assume that Thunderbird is also affected?
|
Assignee | |
Comment 5•18 years ago
|
||
I've been looking at this on the mac, which I think has the same bug. I believe the bug is in the xpcom/obsolete/filestream code, but I'm still debugging it. It's also a core bug, not specific to seamonkey.
|
|
Reporter | |
Comment 6•18 years ago
|
||
Hello,
I was able to obtain a stack trace to where the bug occurs:
(gdb) bt
#0 *__GI___xstat (vers=3, name=0xa5ae800 "/home/testie/.mozilla/testie/seletqfv.slt/Mail/Local Folders/Testfolder", buf=0x7fd4be84)
at ../sysdeps/unix/sysv/linux/i386/xstat.c:92
#1 0x7731ea58 in stat (__path=0xa5ae800 "/home/testie/.mozilla/testie/seletqfv.slt/Mail/Local Folders/Testfolder", __statbuf=0x7fd4be84) at /usr/include/sys/stat.h:436
#2 0x7731a22d in nsFileSpec::Exists (this=0xa5acf8c) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/xpcom/obsolete/nsFileSpecUnix.cpp:162
#3 0x77324efb in nsFileSpecImpl::Exists (this=0xa5acf80, _retval=0x7fd4bf2c) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/xpcom/obsolete/nsFileSpecImpl.cpp:376
#4 0x08f69d2a in nsMsgLocalMailFolder::AddSubfolder (this=0xa56d000, name=@0x7fd4bf64, child=0x7fd4c040)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/mailnews/local/src/nsLocalMailFolder.cpp:281
#5 0x08f6a790 in nsMsgLocalMailFolder::CreateSubFolders (this=0xa56d000, path=@0x7fd4c0dc)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/mailnews/local/src/nsLocalMailFolder.cpp:258
#6 0x08f6ad18 in nsMsgLocalMailFolder::GetSubFolders (this=0xa56d000, result=0x7fd4c160)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/mailnews/local/src/nsLocalMailFolder.cpp:430
#7 0x08e4f380 in nsMsgFolderDataSource::createFolderOpenNode (this=0xa3e58b0, folder=0xa56d028, target=0x7fd4c208)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/mailnews/base/src/nsMsgFolderDataSource.cpp:1460
#8 0x08e54f39 in nsMsgFolderDataSource::createFolderNode (this=0xa3e58b0, folder=0xa56d028, property=0x9839210, target=0x7fd4c208)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/mailnews/base/src/nsMsgFolderDataSource.cpp:1035
#9 0x08e5554c in nsMsgFolderDataSource::GetTarget (this=0xa3e58b0, source=0xa56d000, property=0x9839210, tv=1, target=0x7fd4c208)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/mailnews/base/src/nsMsgFolderDataSource.cpp:439
#10 0x08e63623 in GetTargetHasAssertion (dataSource=0xa3e58b0, folderResource=0xa56d000, property=0x9839210, tv=1, target=0x9839270, hasAssertion=0x7fd4c498)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/mailnews/base/src/nsMsgRDFUtils.cpp:100
#11 0x08e5138e in nsMsgFolderDataSource::DoFolderHasAssertion (this=0xa3e58b0, folder=0xa56d028, property=0x9839210, target=0x9839270, tv=1, hasAssertion=0x7fd4c498)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/mailnews/base/src/nsMsgFolderDataSource.cpp:2258
#12 0x08e516ae in nsMsgFolderDataSource::HasAssertion (this=0xa3e58b0, source=0xa56d000, property=0x9839210, target=0x9839270, tv=1, hasAssertion=0x7fd4c498)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/mailnews/base/src/nsMsgFolderDataSource.cpp:593
#13 0x08841e7c in nsXULTreeBuilder::IsContainerOpen (this=0xa3e1500, aResource=0xa56d000, aOpen=0x7fd4c498)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/xul/templates/src/nsXULTreeBuilder.cpp:1794
#14 0x08842aab in nsXULTreeBuilder::IsContainerOpen (this=0xa3e1500, aResult=0xa5a9d98, aOpen=0x7fd4c498)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/xul/templates/src/nsXULTreeBuilder.cpp:1783
#15 0x08843f8c in nsXULTreeBuilder::OpenSubtreeForQuerySet (this=0xa3e1500, aSubtree=0xa3e1594, aIndex=-1, aResult=0xa5a9bc8, aQuerySet=0xa563720, aDelta=0x7fd4c65c,
open=@0x7fd4c62c) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/xul/templates/src/nsXULTreeBuilder.cpp:1681
#16 0x088441d4 in nsXULTreeBuilder::OpenSubtreeOf (this=0xa3e1500, aSubtree=0xa3e1594, aIndex=-1, aResult=0xa5a9bc8, aDelta=0x7fd4c7b4)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/xul/templates/src/nsXULTreeBuilder.cpp:1538
#17 0x0884449c in nsXULTreeBuilder::OpenContainer (this=0xa3e1500, aIndex=-1, aResult=0xa5a9bc8)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/xul/templates/src/nsXULTreeBuilder.cpp:1511
#18 0x0884517b in nsXULTreeBuilder::RebuildAll (this=0xa3e1500) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/xul/templates/src/nsXULTreeBuilder.cpp:1382
#19 0x0884c2b2 in nsXULTemplateBuilder::Rebuild (this=0xa3e1500)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/xul/templates/src/nsXULTemplateBuilder.cpp:309
#20 0x08851409 in nsXULTemplateBuilder::AttributeChanged (this=0xa3e1500, aDocument=0x9b3b688, aContent=0xa2c6e60, aNameSpaceID=0, aAttribute=0x9842c08, aModType=2)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/xul/templates/src/nsXULTemplateBuilder.cpp:890
#21 0x08758c5a in nsBindingManager::AttributeChanged (this=0x9b3b930, aDocument=0x9b3b688, aContent=0xa2c6e60, aNameSpaceID=0, aAttribute=0x9842c08, aModType=2)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/xbl/src/nsBindingManager.cpp:1173
#22 0x08694620 in nsNodeUtils::AttributeChanged (aContent=0xa2c6e60, aNameSpaceID=0, aAttribute=0x9842c08, aModType=2)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/base/src/nsNodeUtils.cpp:98
#23 0x08680629 in nsGenericElement::SetAttrAndNotify (this=0xa2c6e60, aNamespaceID=0, aName=0x9842c08, aPrefix=0x0, aOldValue=@0x7fd4cb88, aParsedValue=@0x7fd4cc28,
aModification=0, aFireMutation=0, aNotify=1) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/base/src/nsGenericElement.cpp:3253
#24 0x086809c7 in nsGenericElement::SetAttr (this=0xa2c6e60, aNamespaceID=0, aName=0x9842c08, aPrefix=0x0, aValue=@0xa5a9b88, aNotify=1)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/base/src/nsGenericElement.cpp:3177
#25 0x0868785c in nsGenericElement::SetAttr (this=0xa2c6e60, aNameSpaceID=0, aName=0x9842c08, aValue=@0xa5a9b88, aNotify=1)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/html/content/src/../../../base/src/nsGenericElement.h:392
#26 0x0868504f in nsGenericElement::SetAttribute (this=0xa2c6e60, aName=@0xa5a9b70, aValue=@0xa5a9b88)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/base/src/nsGenericElement.cpp:1386
#27 0x08836003 in nsXULElement::SetAttribute (this=0xa2c6e60, name=@0xa5a9b70, value=@0xa5a9b88)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/xul/content/src/nsXULElement.h:580
#28 0x77eca139 in NS_InvokeByIndex () at /root/install/distro/unpackaged/cvs/mozilla/mozilla/xpcom/reflect/xptinfo/src/xptiInterfaceInfo.cpp:73
#29 0x081faf21 in XPCWrappedNative::CallMethod (ccx=@0x7fd4d030, mode=XPCWrappedNative::CALL_METHOD)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/js/src/xpconnect/src/xpcwrappednative.cpp:2162
#30 0x0820858e in XPC_WN_CallMethod (cx=0x9dfea40, obj=0xa3ca1f0, argc=2, argv=0xa567c08, vp=0x7fd4d174)
---Type <return> to continue, or q <return> to quit---
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/js/src/xpconnect/src/xpcwrappednativejsops.cpp:1455
#31 0x77d4ec5f in js_Invoke (cx=0x9dfea40, argc=2, flags=0) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/js/src/jsinterp.c:1396
#32 0x77d6284a in js_Interpret (cx=0x9dfea40, pc=0xa261983 ":", result=0x7fd4de70) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/js/src/jsinterp.c:3948
#33 0x77d4ece9 in js_Invoke (cx=0x9dfea40, argc=1, flags=2) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/js/src/jsinterp.c:1415
#34 0x77d4f07b in js_InternalInvoke (cx=0x9dfea40, obj=0x9e70a08, fval=170339752, flags=0, argc=1, argv=0xa567a38, rval=0x7fd4e08c)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/js/src/jsinterp.c:1490
#35 0x77d0ecdd in JS_CallFunctionValue (cx=0x9dfea40, obj=0x9e70a08, fval=170339752, argc=1, argv=0xa567a38, rval=0x7fd4e08c)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/js/src/jsapi.c:4357
#36 0x0878c2ec in nsJSContext::CallEventHandler (this=0x9e9d008, aTarget=0x9e4c144, aScope=0x9e70a08, aHandler=0xa272da8, aargv=0xa4e0b68, arv=0x7fd4e1f8)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/dom/src/base/nsJSEnvironment.cpp:1750
#37 0x087ec64c in nsJSEventListener::HandleEvent (this=0xa2a4678, aEvent=0x9d9ec68)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/dom/src/events/nsJSEventListener.cpp:212
#38 0x086c5c57 in nsEventListenerManager::HandleEventSubType (this=0x9e090f8, aListenerStruct=0xa2a46a0, aListener=0xa2a4678, aDOMEvent=0x9d9ec68, aCurrentTarget=0x9fe6ba0,
aPhaseFlags=6) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/events/src/nsEventListenerManager.cpp:1266
#39 0x086c6193 in nsEventListenerManager::HandleEvent (this=0x9e090f8, aPresContext=0x9b3c598, aEvent=0x7fd4e5e4, aDOMEvent=0x7fd4e560, aCurrentTarget=0x9fe6ba0, aFlags=6,
aEventStatus=0x7fd4e564) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/events/src/nsEventListenerManager.cpp:1382
#40 0x086e58f0 in nsEventTargetChainItem::HandleEvent (this=0x9c0b828, aVisitor=@0x7fd4e558, aFlags=6)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/events/src/nsEventDispatcher.cpp:355
#41 0x086e5cc5 in nsEventTargetChainItem::HandleEventTargetChain (this=0x9c0b840, aVisitor=@0x7fd4e558, aFlags=6, aCallback=0x0)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/events/src/nsEventDispatcher.cpp:431
#42 0x086e63f0 in nsEventDispatcher::Dispatch (aTarget=0x9fe6ba0, aPresContext=0x9b3c598, aEvent=0x7fd4e5e4, aDOMEvent=0x0, aEventStatus=0x7fd4e63c, aCallback=0x0,
aTargetIsChromeHandler=0) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/content/events/src/nsEventDispatcher.cpp:639
#43 0x08443983 in DocumentViewerImpl::LoadComplete (this=0x9b3bc88, aStatus=0) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/layout/base/nsDocumentViewer.cpp:1068
#44 0x08a86ba7 in nsDocShell::EndPageLoad (this=0x9dfe828, aProgress=0x9dfe83c, aChannel=0x9e090a0, aStatus=0)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/docshell/base/nsDocShell.cpp:4740
#45 0x08a48b2d in nsWebShell::EndPageLoad (this=0x9dfe828, aProgress=0x9dfe83c, channel=0x9e090a0, aStatus=0)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/docshell/base/nsWebShell.cpp:972
#46 0x08a8ee2e in nsDocShell::OnStateChange (this=0x9dfe828, aProgress=0x9dfe83c, aRequest=0x9e090a0, aStateFlags=131088, aStatus=0)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/docshell/base/nsDocShell.cpp:4655
#47 0x08a59bde in nsDocLoader::FireOnStateChange (this=0x9dfe828, aProgress=0x9dfe83c, aRequest=0x9e090a0, aStateFlags=131088, aStatus=0)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/uriloader/base/nsDocLoader.cpp:1235
#48 0x08a5a7ba in nsDocLoader::doStopDocumentLoad (this=0x9dfe828, request=0x9e090a0, aStatus=0)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/uriloader/base/nsDocLoader.cpp:858
#49 0x08a5a996 in nsDocLoader::DocLoaderIsEmpty (this=0x9dfe828) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/uriloader/base/nsDocLoader.cpp:763
#50 0x08a5b057 in nsDocLoader::OnStopRequest (this=0x9dfe828, aRequest=0xa43ef78, aCtxt=0x0, aStatus=0)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/uriloader/base/nsDocLoader.cpp:679
#51 0x080eaab7 in nsLoadGroup::RemoveRequest (this=0x9e89710, request=0xa43ef78, ctxt=0x0, aStatus=0)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/netwerk/base/src/nsLoadGroup.cpp:685
#52 0x083792f0 in imgRequestProxy::RemoveFromLoadGroup (this=0xa43ef78, releaseLoadGroup=1)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/modules/libpr0n/src/imgRequestProxy.cpp:162
#53 0x08379453 in imgRequestProxy::OnStopRequest (this=0xa43ef78, request=0xa43f260, ctxt=0x0, statusCode=0, lastPart=1)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/modules/libpr0n/src/imgRequestProxy.cpp:502
#54 0x08375e2b in imgRequest::OnStopRequest (this=0xa43f300, aRequest=0xa43f260, ctxt=0x0, status=0)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/modules/libpr0n/src/imgRequest.cpp:773
#55 0x08370e87 in ProxyListener::OnStopRequest (this=0xa43f658, aRequest=0xa43f260, ctxt=0x0, status=0)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/modules/libpr0n/src/imgLoader.cpp:866
#56 0x761279e7 in nsJARChannel::OnStopRequest (this=0xa43f260, req=0xa43f948, ctx=0x0, status=0)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/modules/libjar/nsJARChannel.cpp:751
#57 0x080e0f9b in nsInputStreamPump::OnStateStop (this=0xa43f948) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/netwerk/base/src/nsInputStreamPump.cpp:571
#58 0x080e10ae in nsInputStreamPump::OnInputStreamReady (this=0xa43f948, stream=0xa43f9ac)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/netwerk/base/src/nsInputStreamPump.cpp:396
#59 0x77e81dd2 in nsInputStreamReadyEvent::Run (this=0xa3b8548) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/xpcom/io/nsStreamUtils.cpp:111
#60 0x77eb33cc in nsThread::ProcessNextEvent (this=0x9741330, mayWait=1, result=0x7fd4efa4)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/xpcom/threads/nsThread.cpp:482
#61 0x77e3f1b7 in NS_ProcessNextEvent_P (thread=0x9741330, mayWait=1) at nsThreadUtils.cpp:225
#62 0x77eb3850 in nsThread::Shutdown (this=0xa4d3fd8) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/xpcom/threads/nsThread.cpp:442
---Type <return> to continue, or q <return> to quit---
#63 0x77eca139 in NS_InvokeByIndex () at /root/install/distro/unpackaged/cvs/mozilla/mozilla/xpcom/reflect/xptinfo/src/xptiInterfaceInfo.cpp:73
#64 0x77ebcd07 in nsProxyObjectCallInfo::Run (this=0xa4deef8) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/xpcom/proxy/src/nsProxyEvent.cpp:181
#65 0x77eb33cc in nsThread::ProcessNextEvent (this=0x9741330, mayWait=1, result=0x7fd4f0c0)
at /root/install/distro/unpackaged/cvs/mozilla/mozilla/xpcom/threads/nsThread.cpp:482
#66 0x77e3f1b7 in NS_ProcessNextEvent_P (thread=0x9741330, mayWait=1) at nsThreadUtils.cpp:225
#67 0x083cbe2c in nsBaseAppShell::Run (this=0x983bc40) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/widget/src/xpwidgets/nsBaseAppShell.cpp:153
#68 0x08d59466 in nsAppStartup::Run (this=0x9842328) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/xpfe/components/startup/src/nsAppStartup.cpp:218
#69 0x0807a87c in main1 (argc=1, argv=0x7fd4f3b4, nativeApp=0x9746b50) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/xpfe/bootstrap/nsAppRunner.cpp:1245
#70 0x0807ad4b in main (argc=1, argv=0x7fd4f3b4) at /root/install/distro/unpackaged/cvs/mozilla/mozilla/xpfe/bootstrap/nsAppRunner.cpp:1747
Actually, it is a severe design bug. The method nsFileSpec::Exists() returns either "true" or "false", but it may not return the state "unknown" or "error happened during this operation". However, the underlying calls may return something different from both "yes, the file exists" and "no, the file does not exist", like ENOMEM, EACCES, EOVERFLOW. (In Java, we would throw and catch an Exception in these conditions.) Thus, the method nsFileSpec::Exists() erroneously replaces these "I do not know" conditions to "I know that the file does not exist". This, in turn leads seamonkey to try to create the file again, which actually results in overwriting (truncating) the file.
The correct way to fix this bug is to introduce at least another possible result (like "unknown") additionally to "yes, the file exists" and "no, the file does not exist". However, this is an API change, implying that all callers of this method have to be changed. As all callers need a review anyway (checking what they do if stat(3) returns an error condition and thus Exists() errorneously returns false), changing all callers is also the correct (but possibly painful) solution to this severe bug.
|
|
Assignee | |
Comment 7•18 years ago
|
||
thx for the investigation and analysis - WHY is there an error checking if the file exists? The stack trace is on folder discovery, presumably after a restart - my guess is something happened before you restarted that caused an error...
|
|
Reporter | |
Comment 8•18 years ago
|
||
We know for sure that if EOVERFLOW happened, the file existed, just the stat data could not be filled nicely without an overflow. Thus, returning "Yes, the file exists" if stat(3) returns -1, but errno is EOVERFLOW, is the desired behaviour.
However, if errno is one of
EACCES
EIO
ELOOP
ENAMETOOLONG
ENOTDIR
EOVERFLOW
EFAULT
ENOMEM
the result is not "unknown" but remains to be "No, the file does not exist", which is clearly wrong in most cases (though, in some cases like EACCESS, it might not matter much). In cases like EIO, a partly unreadable file might (due to this bug) get fully deleted (truncated) unintentionally, depending on the filesystem.
Only if errno==ENOENT, the method should return "No, the file does not exist".
Attachment #246505 -
Flags: review?
|
|
Reporter | |
Comment 9•18 years ago
|
||
Hello David,
if the file exists, you cannot asume that stat() will return 0. Even if the file exists, stat() might return -1 and set errno to something like EACCES, EIO, ENAMETOOLONG, EOVERFLOW, EFAULT, ENOMEM. Conversely, this is not bijective in every case. So if you get ENOMEM, EIO or EACCES, you do not know wether the file existed or not.
Why seamonkey runs into folder discovery, I do not know. Presumably because the corresponding ".msf" file is empty. But this may happen, e.g. if seamonkey or the machine crashes during the generation of the ".msf" file. Whatever leads to this situation, no mozilla application shall ever overwrite or delete any user's file without the user's intention and confirmation.
|
|
||
Comment 10•18 years ago
|
||
Comment on attachment 246505 [details] [diff] [review]
This is a specific workaround to the specific problem which triggered the bug. However, some (rarer) cases are still unhandled
access(mPath, F_OK); might be an alternative that doesn't have the EOVERFLOW problem for large files (or stat64, of course).
I also want to note that this API (nsFileSpec) is deprecated and that the non-deprecated version (nsIFile/nsILocalFile) does have a way to report errors.
|
|
Assignee | |
Comment 11•18 years ago
|
||
> Presumably because the corresponding ".msf" file is empty
No, we don't care about that - we'll just regenerate it.
So stat() is overflowing because it can't fit the file size in a signed 32 bit int - stat64 sounds like a good choice, and being more discriminating with the stat error codes...
|
|
Assignee | |
Comment 12•18 years ago
|
||
I tried the following:
Index: nsFileSpecUnix.cpp
===================================================================
RCS file: /cvsroot/mozilla/xpcom/obsolete/nsFileSpecUnix.cpp,v
retrieving revision 1.9
diff -u -w -p -r1.9 nsFileSpecUnix.cpp
--- nsFileSpecUnix.cpp 16 Mar 2005 20:07:34 -0000 1.9
+++ nsFileSpecUnix.cpp 27 Nov 2006 18:25:43 -0000
@@ -158,7 +158,7 @@ PRBool nsFileSpec::Exists() const
//----------------------------------------------------------------------------------------
{
struct stat st;
- return !mPath.IsEmpty() && 0 == stat(mPath, &st);
+ return !mPath.IsEmpty() && 0 == access(mPath, F_OK);
} // nsFileSpec::Exists
and I still lost my folder copying messages into a 2GB+ folder. I'm not sure if the access call failed, or if some other code went asunder. I'll debug it further.
|
|
Assignee | |
Comment 13•18 years ago
|
||
switching to core component
Component: MailNews: Main Mail Window → MailNews: Backend
Product: Mozilla Application Suite → Core
Version: unspecified → Trunk
|
|
Assignee | |
Comment 14•18 years ago
|
||
requesting blocking...I'm still investigating this, however...
Flags: blocking1.8.0.9?
|
|
Reporter | |
Comment 15•18 years ago
|
||
Hello David,
- return !mPath.IsEmpty() && 0 == stat(mPath, &st);
+ return !mPath.IsEmpty() && ((0 == stat(mPath, &st)) || (errno==EOVERFLOW));
works for me. Does it work for you, too?
Component: MailNews: Backend → Build Config
Flags: blocking1.8.0.9?
Product: Core → Mozilla Application Suite
Version: Trunk → 1.0 Branch
|
|
Assignee | |
Comment 16•18 years ago
|
||
this really is a core bug - my changes got crunched, apparently. I'll try that specific change, though I suspect that only helps with special folders like Inbox, Sent, etc.
Product: Mozilla Application Suite → Core
|
|
Assignee | |
Updated•18 years ago
|
Assignee: mail → bienvenu
Component: Build Config → MailNews: Backend
Version: 1.0 Branch → Trunk
|
|
Assignee | |
Comment 17•18 years ago
|
||
Xuân, can you try the "access" version that I put in a previous comment? Thx!
Status: NEW → ASSIGNED
Flags: blocking1.8.0.9?
|
|
Reporter | |
Comment 18•18 years ago
|
||
(In reply to comment #16)
> this really is a core bug - my changes got crunched, apparently.
David,
I'm sorry. I did not intend to revert your changes and, at submission time, it was not apparent to me that I actually triggered this revert. This just happened because we together did a kind of parallel submission (we both started from the state of comment #12 and then submitted intependently) and bugzilla did not complain about the risk of overwriting new state by old state due to parallel submission. This is another bugzilla bug.
> I'll try that
> specific change, though I suspect that only helps with special folders like
> Inbox, Sent, etc.
I tested this on non-special (i.e. user defined) mail folders of >2GiB size, and it worked fine there.
|
|
Reporter | |
Comment 19•18 years ago
|
||
(In reply to comment #17)
> Xuân, can you try the "access" version that I put in a previous comment? Thx!
>
David,
I did test your "access" version, and it works well. :-) Maybe you tested this using a wrong (old) binary?
|
|
Assignee | |
Comment 20•18 years ago
|
||
I'm not sure - I haven't been able to reproduce the problem since, with the access() version of the patch (I never got around to trying your patch, since I prefer the access() version...) - so I think your idea is a big improvement, and I'll try to get it landed.
We still shouldn't be limited to 2GB - up to 4GB works on Windows, so I'm going to look at that next...
|
|
Assignee | |
Comment 21•18 years ago
|
||
Attachment #246709 -
Flags: superreview?(mscott)
|
||
Comment 22•18 years ago
|
||
(In reply to comment #21)
> Created an attachment (id=246709)
Couldn't the line "struct stat st;" be removed with this patch, since the struct istn't used anymore?
|
||
Updated•18 years ago
|
Attachment #246709 -
Flags: superreview?(mscott) → superreview+
|
|
Assignee | |
Comment 23•18 years ago
|
||
Comment on attachment 246709 [details] [diff] [review]
biesi's proposal
requesting 1.8.0.9 approval - this affects mac and linux. I think it's fairly safe, and fixes a nasty bug.
Attachment #246709 -
Flags: approval1.8.0.9?
|
|
Assignee | |
Comment 24•18 years ago
|
||
fixed on trunk and 2.0 branch.
|
|
Assignee | |
Comment 25•18 years ago
|
||
the 2GB problem has to do with unix's that don't support 64 bit seeks, or indeed, any seeks > 2GBs. xpcom/obsolete/nsIFileStream.cpp has this problem, but so does netwerk/base/src/nsFileStreams.cpp.
This can be detected with an #ifdef _PR_NO_LARGE_FILES, so I could change the 4GB warning to a 2GB warning on those platforms. But I can't believe Mac and various linuxes don't have some way around this problem that NSPR could use
Component: MailNews: Backend → Build Config
Flags: blocking1.8.0.9?
Product: Core → Mozilla Application Suite
Version: Trunk → 1.0 Branch
|
|
Assignee | |
Comment 26•18 years ago
|
||
argh, I've done it to myself somehow - resetting components again...
Product: Mozilla Application Suite → Core
|
|
Assignee | |
Updated•18 years ago
|
Component: Build Config → MailNews: Backend
Flags: blocking1.8.0.9?
|
|
Reporter | |
Comment 27•18 years ago
|
||
(In reply to comment #25)
> the 2GB problem has to do with unix's that don't support 64 bit seeks, or
> indeed, any seeks > 2GBs. xpcom/obsolete/nsIFileStream.cpp has this problem,
> but so does netwerk/base/src/nsFileStreams.cpp.
>
> This can be detected with an #ifdef _PR_NO_LARGE_FILES, so I could change the
> 4GB warning to a 2GB warning on those platforms. But I can't believe Mac and
> various linuxes don't have some way around this problem that NSPR could use
>
Maybe you can do relative seeks (using SEEK_CUR) in multiple steps, each of these steps in the range of -2GiB..+2GiB ?
Or if we do absolute position, we first seek to the start (using SEEK_END) and then repeatedly advance (using SEEK_CUR)?
|
|
Assignee | |
Comment 28•18 years ago
|
||
yeah, I was thinking of relative seeks as well, if nothing else will work...
|
||
Comment 29•18 years ago
|
||
Comment on attachment 246709 [details] [diff] [review]
biesi's proposal
NSPR also uses access(..., F_OK) to implement
PR_Access(..., PR_Access PR_ACCESS_EXISTS), which checks
the existence of a file.
I don't understand the comments about seeking. There is
no seeking in this patch.
Attachment #246709 -
Flags: review+
|
|
Assignee | |
Comment 30•18 years ago
|
||
wtc - the reference to seeking has to do with a related issue, having to do with mailboxes > 2GB. Basically, on _PR_NO_LARGE_FILES platforms, we can't support mailboxes larger than 2GB because we can't do the needed seeking. It's surprising to me that some unix platforms, including OS/X 10.4, we don't support seeks of > 2GB. As Xuân points out, we could perhaps fake this by breaking up the seeks into offsets < 2GB. We could either do this in various parts of the client code (I know of at least two pieces of client code that would need to change so far) or do it in NSPR. I'm guessing that doing it in NSPR is unlikely, but I thought I'd at least mention it.
Version: 1.0 Branch → Trunk
|
||
Comment 31•18 years ago
|
||
Comment on attachment 246709 [details] [diff] [review]
biesi's proposal
approved for 1.8.0 branch, a=dveditz for drivers
Attachment #246709 -
Flags: approval1.8.0.9? → approval1.8.0.9+
|
|
||
Updated•18 years ago
|
Flags: blocking1.8.0.9? → blocking1.8.0.9+
|
|
||
Updated•18 years ago
|
Whiteboard: needs landing
|
|
||
Comment 33•18 years ago
|
||
We should fix NSPR so that it supports large files on all
major platforms. For Mac OS X, this may just require defining
_PR_HAVE_LARGE_OFF_T instead of _PR_NO_LARGE_FILES in
mozilla/nsprpub/pr/include/md/_darwin.h. (off_t is 8 bytes
on Mac OS X.) Then you can use PR_GetFileInfo64, PR_GetOpenFileInfo64,
and PR_Seek64.
|
|
Assignee | |
Comment 34•18 years ago
|
||
oh, even better - we'd love that for TB 2.0, if possible.
|
|
||
Comment 35•18 years ago
|
||
Please still look into changing the client code just in case I don't
have time to work on NSPR before TB 2.0 is released.
Here are the supported Unix platforms on which _PR_NO_LARGE_FILES is defined:
FreeBSD
Mac OS X
NetBSD
OpenBSD
OpenVMS
QNX / NTO (Neutrino)
RISC OS
Which of these matter to you?
I don't know the status of OS/2 and BeOS because the NSPR code for these
two platforms doesn't use the _PR_NO_LARGE_FILES macro. Does anyone know
if NSPR supports large files on OS/2 and BeOS?
|
|
Assignee | |
Comment 36•18 years ago
|
||
Mac OS/X matters the most - I don't really know about the rest - I'd say the BSD's are probably next in importance, but that's just a guess.
|
|
Reporter | |
Comment 37•18 years ago
|
||
(In reply to comment #35)
> Please still look into changing the client code just in case I don't
> have time to work on NSPR before TB 2.0 is released.
>
> Here are the supported Unix platforms on which _PR_NO_LARGE_FILES is defined:
>
> FreeBSD
> Mac OS X
> NetBSD
> OpenBSD
> OpenVMS
> QNX / NTO (Neutrino)
> RISC OS
>
> Which of these matter to you?
>
> I don't know the status of OS/2 and BeOS because the NSPR code for these
> two platforms doesn't use the _PR_NO_LARGE_FILES macro. Does anyone know
> if NSPR supports large files on OS/2 and BeOS?
>
Are we still talking about this reported bug? Because this bug happened under Linux, which has been supporting stat64() and other 64-bit file operations for years. So, as _PR_NO_LARGE_FILES is not defined for Linux, why could this bug then happen? Or is _PR_NO_LARGE_FILES also defined for Linux?
|
|
||
Comment 38•18 years ago
|
||
I opened NSPR bug 362244 for the NSPR's lack of large file support
on some platforms. Let's stay focused on the original problem in
this bug report.
|
|
Assignee | |
Comment 39•18 years ago
|
||
for QA/testing, this change affects most of the code that checks to see if a file exists in mail/news. But, if it didn't work, mail pretty much wouldn't work. To recreate the bug, you need to create a very large folder, > 2GB, and see if it survives restart on a platform like OS/X.
|
||
Comment 40•18 years ago
|
||
Unfortunately I don't have a folder that large to test - Reporter, can you help verify the bug using the candidate build: ftp://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/2.0.0.0-candidates.
|
||
Updated•18 years ago
|
Attachment #246505 -
Attachment is obsolete: true
Attachment #246505 -
Flags: review?
|
||
Updated•16 years ago
|
Product: Core → MailNews Core
|
|
Reporter | |
Comment 41•16 years ago
|
||
This bug still persists in seamonkey 1.1.12. :-( My sister just lost 6 years worth of e-mail due to this bug. :-(
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
|
||
Comment 42•16 years ago
|
||
David, is there a unit test for this? Xuan, are you perhaps not seeing the exact symptoms of this bug, but rather bug 362244?
(resetting QA contact, was empty)
QA Contact: backend
|
|
Reporter | |
Comment 43•16 years ago
|
||
(In reply to comment #42)
> Xuan, are you perhaps not seeing the
> exact symptoms of this bug, but rather bug 362244?
I do see completely lost Inbox folders, repeatedly. :-( The data loss one happened to a friend about 1 month ago.
|
||
Comment 44•14 years ago
|
||
no activity for a long time, please close
|
||
Comment 45•13 years ago
|
||
Closed per Comment 44
Status: REOPENED → RESOLVED
Closed: 18 years ago → 13 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•