Re: [PATCH 2/2] nnmaildir: Use a 'num' file, instead of a directory

[Paul Jarc]

Ted Zlatanov <address@hidden> wrote:
> On Thu, 27 May 2010 15:09:08 -0600 John Wiegley <address@hidden> wrote: 
>
> JW> This patch causes nnmaildir to use a "num" file to track the next
> JW> available article number, rather than creating N empty files.
> JW> However, this only works for new groups.  Existing groups, which use
> JW> the directory, will continue to.  If you want to switch to using
> JW> file-based numbering, you must convert your Maildir using the
> JW> attached script (it's also copied to the commit description in the
> JW> patch).
>
> IIRC (and Paul Jarc may want to comment) something similar was discussed
> in http://thread.gmane.org/gmane.emacs.gnus.general/66245/focus=66247

Yes, although John hasn't said whether performance is his motivation.

> I have recently experienced slow nnmaildir performance

I suspect the performance problem is with the nov/ directory, rather
than num/.  With the current nov/ structure, it's quick and easy to
map from filenames to article numbers, but I think we usually need the
inverse operation.  As it is, to map from an article number to a
filename, we need to read the contents of all the nov/ files.  That's
done just once and the results are cached, so it's not too horrific,
but we still need to check timestamps to see if the files have
changed, and it takes a lot of memory for large groups.

It's been a while since I looked at it, though--there may be some
operations where we do need to go from the filename to the number.  So
then it might be useful to add hard links so each nov/ file could be
accessed by either its article number or filename.  The filename would
also have to be added to the contents of those files somehow.


paul