Re: [lwip-devel] Packet queues (was Re: [bug #11400] ARP multi-packet-queue modifies TCP ...)

[Jonathan Larmour]

Kieran Mansley wrote:
> On Thu, 2007-04-05 at 15:57 +0100, Jonathan Larmour wrote:
> > So is zero copy at present impossible to implement safely in lwIP?
>
> Probably, yes.  I had wondered the same thing.
>
> > That's 
> > bad if so, as one of the good things about zero copy is not just 
> > performance but memory use. This is very wasteful given there being plenty 
> > of DMAing controllers out there now capable of zero copy.
>
> True, but there are very few standard APIs that are capable of it, which
> is probably why there hasn't been much concern about it.

But I had thought it was meant to be a feature of the lwip API :).

> Zero-copy
> through the sockets API is nigh on impossible for example.

That's why the sockets API isn't very good for low memory use. But even 
there it is valuable - otherwise you risk copying not just once but twice: 
once in lwip, and once in the driver because the driver doesn't know where 
the data came from and so whether the data is safe to use directly or not 
(is it a PBUF_RAM from within the stack or within the application?).

> > If this is the case, then I think we're going to have to improve the API.
>
> For the raw API I think this would be wise.
>
> > Rather than trying to retain the copyable state through an API parameter 
> > passed all the way through (like the copy flag to tcp_write), or break the 
> > API, it may be better to add a new pbuf type PBUF_RAM_NOCOPY.
>
> I think I'd prefer to keep it symmetric with the way that TCP does it at
> the moment - ie. use a copy flag.  This would mean that the existing
> mode of operation would be preserved as it would be an extension rather
> than a change in behaviour, although the udp_send() function would
> require an extra parameter. Having both tcp_send() and udp_send() work
> in the same way is attractive.

This requires quite a lot of disruptive changes inside lwIP though - right 
now the copy flag to tcp_write only indicates how to create a TCP seg. But 
as we've seen, it isn't propagated elsewhere in the stack, e.g. through to 
the IP layer, the netif layer, etharp, and then we have to change the 
interface to all drivers (linkoutput) - every function needs the new 
argument. That's a lot of change.

A new pbuf type seems the most backwardly compatible approach as no 
external APIs change, and drivers don't even have to support it: the user 
can ensure they only use it if they know the driver supports it. The notion 
of whether that pbuf needs copying or not stays with the pbuf all the way 
through the stack with no further changes. This approach is also amenable 
to being a configurable option since it should be well contained. Pretty 
much the only special treatment for PBUF_RAM_NOCOPY would be in the low 
level drivers themselves, which would call pbuf_free() on tx completion for 
this type of pbuf.

Note that tcp_write is already assymetric with udp_send: the former takes a 
character array and length, the latter takes a pbuf instead of those args. 
This change would only mean the pbuf also subsumes the 'copy' property as well.

Jifl
--
eCosCentric Limited      http://www.eCosCentric.com/     The eCos experts
Barnwell House, Barnwell Drive, Cambridge, UK.       Tel: +44 1223 245571
Registered in England and Wales: Reg No 4422071.
  **  Visit us at ESC Silicon Valley <http://www.embedded.com/esc/sv> **
  **  April 3-5 2007, Booth 1922, San Jose McEnery Convention Center  **
------["The best things in life aren't things."]------      Opinions==mine