[Nmh-workers] Oinkage in the 'post' command SSL handling...

[Valdis . Kletnieks]

So I had occasion to post an email that contained a data file, about 1M in
size, averaging maybe 15 bytes/line (it was basically e-mail addresses from an
extract, one per line).  I hit 'send', and I'm watching gkrellm report 300K/sec
going out my cable connection to our mail hub.. and after 3 seconds it's not
done sending.  After a minute it's still blasting at 300K/sec.  6 minutes or so
later, it finally finishes.

I found this piece of info: 
http://www.imperialviolet.org/2010/06/25/overclocking-ssl.html

"Packets and records

SSL/TLS packages the bytes of the application protocol (HTTP in our case) into
records. Each record has a signature and a header. Records are packed into
packets and each packet has headers. The overhead of a record is typically 25
to 40 bytes (based on common ciphersuites) and the overhead of a packet is
around 52 bytes. So it's vitally important not to send lots of small packets
with small records in them.

I don't want to be seen to be picking on Bank Of America, it's honestly just
the first site that I tried, but looking at their packets in Wireshark, we see
many small records, often sent in a single packet. A quick sample of the record
sizes: 638 bytes, 1363, 15628, 69, 182, 34, 18, \u2026 This is often caused
because OpenSSL will build a record from each call to SSL_write and the kernel,
with Nagle disabled, will send out packets to minimise latency.

This can be fixed with a couple of tricks: buffer in front of OpenSSL and don't
make SSL_write calls with small amounts of data if you have more coming. Also,
if code limitations mean that you are building small records in some cases,
then use TCP_CORK to pack several of them into a packet."

I do some back-of-envelope math, and even at each 15 byte line becoming a 
one-record
packet, that's still 15+40+50 bytes - call it 100, or a factor of  around 10.  
So what would
have taken 3 seconds at line speed would take 30 seconds.  But mine took around
6 minutes.

Are we sending packets that have single 1-byte records in them?  1+40+50 is 
about
the only way I can come up with the bloat factor that I'm seeing...

pgpfACYJH08u3.pgp
Description: PGP signature