Re: [Qemu-devel] Re: [PATCH 2/3] virtio network device

[Jamie Lokier]

Paul Brook wrote:
> > > > virtio makes things a bit trickier though.  There's a shared ring queue
> > > > between the host and guest.  The ring queue is lock-less and depends on
> > > > the ability to atomically increment ring queue indices to be SMP safe.
> > > > Using a copy-API wouldn't be a problem for QEMU since the host and
> > > > guest are always running in lock-step.  A copy API is actually needed
> > > > to deal with differing host/guest alignment and endianness.
> > >
> > > That seems a rather poor design choice, as many architectures don't have
> > > an atomic increment instruction. Oh well.
> >
> > Most have compare-and-swap or load-locked/store-conditional
> > instructions, though, which can be used to implement atomic increment.
> 
> Yes, but your "hardware" implementation has to make sure it interacts with 
> those properly. It's certainly possible to implement lockless lists without 
> requiring atomic increment. Most high-end hardware manages it and that 
> doesn't even have coherent DMA.

I'm inclined to agree that a lockless structure (including not using
an atomic op) for the most commonly used paths, such as appending to a
ring, would be better.  It increases the potential portability for
emulation/virtualisation across all architectures now and in the
future, and it would almost certainly perform better on architectures
other than x86.

However, occasionally you need to enter the host for synchronisation.
E.g. when a ring is empty/full.

In that case, sometimes using atomic ops in the way that futexes are
used in Linux/Glibc can optimise the details of those transitions, but
it would be best if they were optional optimisations, for
cross-platform, cross-architure portability.

There's a particularly awkward problem when taking an x86 atomic op in
the guest, and generating code on the non-x86 host which doesn't have
any equivalent op.  What's the right thing to do?

Since virtio is driven by virtualisation projects rather than
emulation, is it possible this hasn't been thought of at all, making
virtio unusable for cross-architecture emulation?  That would be
really unfortunate.

-- Jamie