Re: [Qemu-devel] [PATCH RFC] virtio: put last seen used index into ring itself

[Avi Kivity]

On 05/20/2010 01:33 AM, Michael S. Tsirkin wrote:
> > Virtio is already way too bouncy due to the indirection between the
> > avail/used rings and the descriptor pool.  A device with out of order
> > completion (like virtio-blk) will quickly randomize the unused
> > descriptor indexes, so every descriptor fetch will require a bounce.
> >
> > In contrast, if the rings hold the descriptors themselves instead of
> > pointers, we bounce (sizeof(descriptor)/cache_line_size) cache lines for
> > every descriptor, amortized.
> >      
> On the other hand, consider that on fast path we are never using all
> of the ring. With a good allocator we might be able to keep
> reusing only small part of the ring, instead of wrapping around
> all of it all of the time.
>    

It's still suboptimal, we have to bounce both the avail/used rings and 
the descriptor pool, compared to just the descriptor ring with a direct 
design.  Plus we don't need a fancy allocator.

When amortizing cachelines, simple data structures win.

--
Do not meddle in the internals of kernels, for they are subtle and quick to 
panic.