Re: [lwip-devel] Best method to allocate space for external packets

[Jonathan Larmour]

Thomas Taranowski wrote:
> I have a scenario where I want to allocate a pbuf for an incoming 
> packet, but I want that pbuf payload to reside in a region of shared 
> memory that is specified at runtime, and is external to the network 
> stack.  This newly allocated pbuf is then passed to the driver, which 
> then DMAs an incoming payload directly into the space.
>
> So, to break it down, I want to make the pbuf framework allocate space 
> from an external pool for me, and I'd like to make the mechanism generic 
> enough so that it will be of general use.
>
> I'm trying to sort out which of the following methods would make the 
> best sense...

First of all, be aware that there's a big difference between the pbuf pool 
in CVS and what is in earlier releases. The pool is now allocated from 
memory using memp_malloc.

I have been thinking about a similar sort of thing for trying to reduce 
the number of copies in lwIP. In particular, if you are using a MAC 
capable of DMA, then there is often constraints such as alignment of the 
destination buffer, or possibly location in the address space (a PCI 
window, say). Right now for many devices, there's no choice but to copy.

> Use the PBUF_ROM flag. 
> When doing the pbuf_alloc with a PBUF_ROM flag set, have the PBUF_ROM 
> case allocate from an external pool.  However, the method to use for the 
> actual allocation is unclear in my mind.  I'm thinking perhaps a new
> MEMP type that refers to an external pool(not part of the memp static 
> buffer), or maybe a mem_malloc that can allocate from an external pool.  
> The alternative would be to implement a custom allocator for the pool, 
> but it seems like I should be able to take advantage of the allocation 
> mechanisms already built-in.

That seems like an uncomfortable overloading of the purpose of PBUF_ROM. 
It would be better to use a different PBUF type if considering that sort 
of approach surely?

> Allocate externally.
> I could allocate the necessary space external to the stack, then use 
> pbuf_ref to refer to the payload.  However, I'd prefer using a 
> pbuf_alloc approach, primarily because the code is already there.

The problem with pbuf_ref is that you won't know when you can free the 
payload.

> Memcpy.
> I could also allocate the space externally, then memcpy the data to a 
> PBUF_POOL.  I don't really care for this method because it adds an 
> additional layer of memcpy that I don't need, but it has the advantage 
> of being easy.

I'm sure this is what many people do at present in that sort of situation.

I'm not sure whether you need _every_ incoming packet to go into this 
shared memory area, or just some.

What I had been thinking personally was to change pbuf.c so that the line:
      q = memp_malloc(MEMP_PBUF_POOL);
becomes
      q = PBUF_POOL_MALLOC();

which will be defined as:
#ifndef PBUF_POOL_MALLOC
# define PBUF_POOL_MALLOC() memp_malloc(MEMP_PBUF_POOL)
#endif

but can be changed by the port to malloc memory from elsewhere. Ditto for 
the memp_free(MEMP_PBUF_POOL,p) later on.

I don't think we can try and fit anything into the existing memp allocator 
system, since it's inherent in the design of that that all the pools are 
contiguous and sequential.

The simplification here is that _all_ pool pbufs would then be allocated 
like that. This is fine most of the time but could be more complicated if 
there are multiple devices, not all of which need this system. Should we 
be worried about that? I don't know but from my thinking about it, it 
would be a very hard problem to solve without breaking abstractions and/or 
lots of APIs. Maybe we should leave it to the port and driver.

Thoughts?

Jifl
--
eCosCentric Limited      http://www.eCosCentric.com/     The eCos experts
Barnwell House, Barnwell Drive, Cambridge, UK.       Tel: +44 1223 245571
Registered in England and Wales: Reg No 4422071.
------["The best things in life aren't things."]------      Opinions==mine