Closed Bug 101273 Opened 23 years ago Closed 6 years ago

Alternative to mbox storage format to avoid "compact folders" and speed up message processing

Categories

(MailNews Core :: Backend, enhancement)

x86
Linux
enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 845952

People

(Reporter: dahechler, Assigned: mscott)

Details

(1) Each folder be a separate directory. (2) With each folder have a status file that is nothing more than a list of message "names" (3) Each message comes in and is stored as a separate file. Inbox ---------------------------- | | | | status file msg1 msg2 msgX So, for the folder "Inbox" (1) create a directory "Inbox" (2) create a "index file" in the Inbox directory --------------------------------- (1) As each email come in, store it in it's own file within the Inbox directory (2) update the index file with the email name by appending the email name at the end of the list ------------------------------ (1) As each email is deleted or moved, delete the email file from the Inbox directory (2) Delete the email name from the index file and delete the empty slot (compressing the file) Savings: (1) This will completely eliminate the need to "compact folders" (2) Since each email is it's own file, it makes it much easier to move the email around from folder to folder and delete.
Not sure why this was filed under editor core ... over to MailNews
Assignee: kin → mscott
Component: Editor: Core → Mail Back End
Product: Browser → MailNews
QA Contact: sujay → esther
Or at least offer the option to TURN OFF comapct folder. I'm using NTFS filesystem comression and that is sufficient enough for me.
Reporter : storing each mail as an individual file is a very bad idea when there are more than a few hundred files involved, especially if the files are small. For instance : my 20GB Fat32 hard drive has 16kb allocation units. This is quite normal. Every file will occupy a multiple of 16kb on the disk. A 29kb file will occupy 32kb, a 33kb file will occupy 48kb, etc. More importantly, a 1kb file (like, say, an e-mail) will occupy 16 kb. 2000 2kb files will occupy approx 32000kb. 32 Megabytes of disk space to store 4 Megabytes of data. This is not very efficient. This can be a real issue with newsgroups, where it is not unusual to have several thousand messages on disk at once for some groups. In addition, once you get over several thousand files in one directory, the OS itself will start to creak. Have you ever tried to browse a directory with several hundred thousand files in it? I have. The OS shells involved (WinNT4 and Win2k) didn't like it - the systems hung. They might have started moving again once they'd finished processing the file list, but after 20 minutes I gave up waiting. It's not a good idea to leave that kind of trap lying around for users to blunder into. Marking WONTFIX
Status: UNCONFIRMED → RESOLVED
Closed: 23 years ago
Resolution: --- → WONTFIX
Oh well, it sounded good to me since the mainframe world has been doing it for over 20 years (before PC's). At McDonnell Douglas, we had the whole campus on mainframe email (PROFS) which handled 500,000 employees !!
Juicy tidbit of info: Meijer (http://www.meijer.com) still uses PROFS for their large grocery team's communications system. Adding David, but I think this will just get verified as won'tfix.
yeah, won't fix - we use berkeley mailbox format because it's a standard, and the benefits outweigh the drawbacks. And switching would be an large undertaking.
verified.
Status: RESOLVED → VERIFIED
*** Bug 133172 has been marked as a duplicate of this bug. ***
Product: MailNews → Core
Product: Core → MailNews Core
This would be solved by bug 845952
Status: VERIFIED → RESOLVED
Closed: 23 years ago6 years ago
Resolution: WONTFIX → DUPLICATE
Summary: Eliminate the need to "compact folders" and speed up message processing → Alternative to mbox storage format to avoid "compact folders" and speed up message processing
You need to log in before you can comment on or make changes to this bug.