[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Nmh-workers] Slowdown with a Populated Folder.
From: |
David Levine |
Subject: |
Re: [Nmh-workers] Slowdown with a Populated Folder. |
Date: |
Tue, 31 Dec 2013 14:44:42 -0500 |
Ralph wrote:
> My rmmproc creates filenames like ,10139,1388316255.094920554
> so as more of those pile up, less entries fit in the 32KiB and
> more calls to it are needed and there's more entries starting
> with `,' to be skipped over. Clearly, I need to switch to ','
> + base254-encoding. ;-)
Or move the ',' files to a separate directory? That would just
delay your problem, but maybe significantly?
> A couple of points come from it... Why does mark(1) above have to read
> the folder when it's just doing a list of the .mh_sequences?
Well, it doesn't have to, but this call sequence is used in about
30 programs:
call folder_read()
call m_convert() to parse message ranges and sequences
including mark(1), which then goes on to add, delete, and/or list as
requested.
If we had to do it all over again, we'd likely come up with a
different folder data structure that better favors time over
space. That would be an interesting project, but I'm not about
to take it on.
> Is it verifying the sequence's content against what still sits in
> the folder?
It is, given the sequence above. If we expect that most
sequences have small numbers of messages, then there's
clearly a more efficient way.
> `mark -s foo -d all' gives an error if sequence foo
> doesn't exist, at least with this old version.
And it still does. mark hasn't changed significantly since
1.3 (and beyond, I expect).
> Consequently, another mark is needed first, the one above, to
> detect if the sequence exists. A more useful interface would
> cut the number of userspace commands that need to run, and
> thus the number of getdents.
Or is there a way to deal gracefully with the error if
sequece foo doesn't exist?
David