Re: [Nmh-workers] MH + IMAP? (was Large MH directories

[Ken Hornstein]

>scan +big-imap-folder last:200
>
>You're gonna notice.

If you look at the IMAP protocol, there are a ton of optimizations you
can do.  For example, you could just extract the information you care
about.  For example, this could be:

C: A001 SELECT archive
S: * 40000 EXISTS
S: A001 OK SELECT completed
C: A001 FETCH 38001:40000 (FLAGS RFC822.SIZE BODY[HEADER.FIELDS (DATE FROM TO 
SUBJECT BODY[]<0..80>)
S: <whole lot of data>

Forgive me if I get the syntax wrong here; I think it's something close
to that.  The idea is that you'd only get the stuff you care about for
the average scan format.  This being the IMAP flags (for %{replied}),
the message size, the headers you asked about in your scan format, and
the first 80 bytes of the bodu of the message.  Since we know what
headers are asked for when we compile a format string it's easy to
generate a list of "interesting" headers.  This would let you push off a
lot of the I/O onto the IMAP server, which for a larger folder could be
a win.

The hard part I see here (well, okay, there are a BUNCH of hard parts)
is this requires a complete rethink of internal nmh API.  Right now a
simple implementation would FETCH a whole message, parse the message,
spit out the results, then move onto the next message.  That would suck.
But the protocol allows us to be much smarter.

--Ken