Re: [Qemu-devel] [PATCH v2] netmap backend (revised)

[Stefan Hajnoczi]

On Wed, Jan 23, 2013 at 12:50:26PM +0100, Luigi Rizzo wrote:
> On Wed, Jan 23, 2013 at 12:10:55PM +0100, Stefan Hajnoczi wrote:
> > On Tue, Jan 22, 2013 at 08:12:15AM +0100, Luigi Rizzo wrote:
> The debugging macros (D, RD() )  are taken from our existing code,
> > 
> > > +#define ND(fd, ... )     // debugging
> > > +#define D(format, ...)                                          \
> ...
> > > +/* rate limited, lps indicates how many per second */
> > > +#define RD(lps, format, ...)                                    \
> ...
> 
> > Have you seen docs/tracing.txt?  It can do fprintf(stderr) but also
> > SystemTap/DTrace and a simple built-in binary tracer.
> 
> will look at it. These debugging macros are the same we use in other
> netmap code so i'd prefer to keep them.

Okay.  Feel free to leave them.

> > > +// a fast copy routine only for multiples of 64 bytes, non overlapped.
> > > +static inline void
> > > +pkt_copy(const void *_src, void *_dst, int l)
> ...
> > > +                *dst++ = *src++;
> > > +        }
> > > +}
> > 
> > I wonder how different FreeBSD bcopy() is from glibc memcpy() and if the
> > optimization is even a win.  The glibc code is probably hand-written
> > assembly that CPU vendors have contributed for specific CPU model
> > families.
> > 
> > Did you compare glibc memcpy() against pkt_copy()?
> 
> I haven't tried in detail on glibc but will run some tests.  In any
> case not all systems have glibc, and on FreeBSD this pkt_copy was
> a significant win for small packets (saving some 20ns each; of
> course this counts only when you approach the 10 Mpps range, which
> is what you get with netmap, and of course when data is in cache).
> 
> One reason pkt_copy gains something is that if it can assume there
> is extra space in the buffer, it can work on large chunks avoiding the extra
> jumps and instructions for the remaining 1-2-4 bytes.

I'd like to drop this code or at least make it FreeBSD-specific since
there's no guarantee that this is a good idea on any other libc.

I'm even doubtful that it's always a win on FreeBSD.  You have a
threshold to fall back to bcopy() and who knows what the "best" value
for various CPUs is.

Stefan