Re: [Qemu-devel] [RFC PATCH 0/2] port network layer onto glib

[Paolo Bonzini]

Il 13/03/2013 18:06, mdroth ha scritto:
> But isn't there also an effort to make virtio-blk/virtio-net a model for
> threaded devices/subsystems in general, as opposed to "accelerators" for
> specific use-cases like tap-based backends? I think this is the main
> question, because most of the planning seems contingent on that. And it
> seems to me that if the answer is no, then we need to consider the fact
> that vhost-net seems to serve this purpose already.

The answer is yes, no doubt.  And also as playgrounds for making the
backends thread-safe.  Of course it is okay to start them as
accelerators for specific use-cases, as long as:

* there is a plan for a final design that works for all back-ends.  With
block we were really lucky and each single step provided new features;
but we can still do the same with other subsystems to if we have an idea
of how to get there.

* the amount of duplicated code is limited to the minimum (and hopefully
there is a plan to eradicate duplication altogether).  Duplicated code
between non-threaded and threaded version is not bad; duplicated code
across subsystems is much worse.

> If the answer is yes, don't we also need to look at things like interaction
> between slirp and a threaded network device? Based on comments in the
> other thread, I thought it was agreed that slirp was a particular example
> for something that should be rolled into a GMainContext loop as opposed
> to an AioContext based one?

Slirp is a mess.  Right now it's neither AioContext nor GMainContext.
But GMainContext is not a silver bullet, because on Win32 it only takes
HANDLEs (just like AioContext), not sockets.  GIOChannel is where glib
has the code to provide HANDLEs for sockets (IIRC it uses a buffer that
is managed from a separate thread).  Both solutions are fine:

* add sockets to AioContext.  Not really network-specific, as it would
enable network block backends (NBD, Curl, ...) on Win32.

* use a GMainLoop for the virtio-net-dataplane event loop.  Convert
Slirp to use GIOChannels and add an AioContext (as a GSource) for the
virtio stuff.

I think the first is easier, also because a large part of the code is
already available in main-loop.c.

> To me this suggests that some event loops will ultimately drive
> GMainContext handlers in addition to AioContexts (with the latter perhaps
> being driven at a higher priority with PI mutexs and whatever else that
> entails).

It's possible.  I cannot yet say.

The important point is that glib does not have any magic built in.  The
risk is to confuse NIH with code in need of refactoring.  If we have
NIH, we change to use some other library.  If code is ugly and ad-hoc,
we refactor it and then treat it as just another reusable component.
AioContext is the latter, or at least that was my plan. :)

> What would be nice is if the difference between the iothread's event
> loop and a dataplane (or QMP/VNC/etc) thread's event loop was simply the
> set of AioContexts/GMainContexts that it drives.

100% agreed so far (except s/GMainContexts/GSources)...

> We could do that purely
> with AioContexts as well, but that rules out a large class of
> backends that offloaded event loops can interact with, such as Chardevs,
> so I think modelling how to handle both will provide a threading model
> that scales better with other devices/subsystems.

.. but I think the "no magic" argument applies here too.  After all we
only have a handful of subsystems.  If chardevs are not performance
critical, they can keep running in the main thread.

If one day we find out that we need a real-time serial port, and glib
just doesn't cut it, we shouldn't be ashamed of ripping GIOChannels out,
hand-writing the same stuff, and using a dedicated AioContext.  Of
course by the time we get there we'll have unit tests/qtests to make
sure we do not regress.  Right??? :)

Paolo