Re: [Nmh-workers] IMAP testing, again

[Ken Hornstein]

>> BTW, it turns out it takes on average 0.66 seconds to append a message
>> to a Gmail mailbox
>
>So refiling the half a dozen emails that are a thread I've just finished
>with would be about six seconds because that's appending?  (I don't know
>IMAP.)

Well, "it depends".

APPEND means "add a new message to the mailbox", and after you send that
command you send the message.  If you want to do a refile BETWEEN IMAP
mailboxes, then you could use MOVE IF your IMAP server supports that
(Gmail does), or you could use COPY and then delete the old message.
So a hypothetical refile beween a MH store and a IMAP server would
require the use of APPEND, but between two IMAP mailboxes it wouldn't.
When I did some tests with COPY, it was fast.  It looks like the delay is
bringing into Gimap; once it's in things go pretty quickly.

>> (tls-encrypted) => A2 SELECT "Enron"
>
>........................................................................
>
>> (tls-decrypted) <= * 480832 EXISTS
>
>........................................................................
>
>> Command (SELECT) execution time: 4.457428 sec
>>
>> I don't even have an idea how long it would have taken for nmh to do a
>> readdir() on a directory with that many files.
>
>On my normal slow machine on ext4, reading the current directory of `.',
>`..', and files {1..480832} takes
>
>    $ time ls -f >/dev/null
>
>    real 0m1.184s user 0m0.599s sys 0m0.585s $

Hm, the best I can get here is:

/usr/bin/time ls -f > /dev/null
        9.11 real         0.47 user         0.75 sys

I'm not sure if that's because this MacOS X machine is older, has an
older OS, or it just doesn't cache dirents as much.

>> Strangely, the speed of adding messages to that mailbox seemed to not
>> depend on the number of messages in the mailbox
>
>You're thinking it would be O(n log n) or similar so you'd see an
>effect?

Yeah, I was thinking there would be some per-folder metadata to slow it
down.

>Google is indexing the emails and the search is consulting that index.
>We could do the same, or the user could use one of the existing email
>indexers on their ~/mail, or we could integrate one of those if it's
>installed.

Sure, we COULD do that, or let Google do it for us :-)

Anyway, I felt the data was interesting.  It sort of suggests that on some
IMAP servers there is a fall-off point where the mailbox gets rather
large and painful to deal with.  A caching daemon that had one connection
per mailbox would be a win in those cases.

--Ken