Re: [lwip-users] Re: Long(er) delay to PC ACK when serving images

[Kieran Mansley]

On Mon, 2006-10-09 at 14:21 +0000, Alan Lamphier wrote:

> A max packet size of 730 bytes when running TCP_MSS = 1460 seemed too related 
> to
> be a coincidence, but I still can't find the dependencies to get my image 
> packet
> sizes back up to max.
> 
> Thanks for the leads and any other ideas one might have,

You've discovered that TCP, in an attempt to be more efficient (and this
isn't lwIP specific - in this case it is the behavior of whatever you're
talking to) waits for two packets to be received before sending an
acknowledgement.  It delays the ACK for the first packet in the hope
that there will be some data flowing in the return direction that the
ACK can piggyback on, and so reduce the number of packets being sent.
In most cases this is good.  So as to avoid the ACK never being sent, it
has a timeout that will send it even if there has been no other traffic
in the mean time.  By splitting your 1460-byte packet into two 730 byte
packets you're prompting it to ACK them immediately (it's received two
packets rather than one) and this is why it goes faster.  This is all
something of a simplification (for example some stacks base the
frequency of ACKs on bytes received rather than number of packets), but
illustrates the point well enough.

Normally this is not a problem as the TCP window is hoped to be large
enough that you have many more than two packets in flight at any one
time, and so your sender is not sitting waiting for the ACK for the
first packet.  However, at times when a very small window is configured
(either because you've set it that way in the lwipopts file, or in the
slow-start or congestion avoidance phase of a TCP connection) it will be
a limiting factor.  There is not a lot you can do about this other than
ensure your receive window and send queue length are configured so that
you can have more than one MTU in flight at any one time.  I suppose
that configuring a smaller MTU would be just as good.  i.e. To get best
performance TCP assumes that the send queue length and receive window
are both much more than a single MTU.  It will work even if this isn't
true, but just slowly as you have discovered.

Hope that helps,

Kieran