Re: [Qemu-devel] Modularizing QEMU RFC

[Marc Marí]

On Mon, 3 Aug 2015 10:23:37 +0100
"Daniel P. Berrange" <address@hidden> wrote:

> On Fri, Jul 31, 2015 at 05:45:42PM +0200, Marc Marí wrote:
> > Hi everyone
> > 
> > I propose improving the current modular driver system for QEMU so it
> > can benefit everybody in speed and flexibility. I'm looking for
> > other ideas, comments, critics, etc.
> > 
> > - Background -
> > In order to speed up QEMU, I'm looking at the high number of
> > libraries and dependencies that it loads. I've generated a QEMU
> > image that needs 145 shared libraries to start, and there were
> > still some options disabled. This is a lot, and this means that it
> > is slow.
> > 
> > So I've been looking at actual module system. Yes, QEMU does have a
> > module system, but disabled by default. The problem is, the modules
> > get loaded always during startup. This means, booting with modules
> > enabled is even slower, because loading at runtime is slower that
> > letting the linker do all the work at the start. At this point, I
> > doubt of the benefits of the actual modular system.
> > 
> > But, if disabling the actual block modules (iscsi, curl, rbd,
> > gluster, ssh and dmg) gives 40 ms of speedup, I think is worth an
> > effort of improving modules, and modularizing new parts
> > 
> > - Current module flow -
> > The current module system is based on shared libraries. Each of
> > these libraries has a constructor that registers the BlockDriver
> > structure to the global bdrv_drivers list. This list is later
> > searched by the type, the protocol, etc.
> > 
> > - Proposals -
> > I have in mind two ideas, which are not mutually exclusive:
> 
> [snip]
> 
> > - My comments on my proposals -
> > Ideally, I'd like a mixed solution. The user can specify what wants
> > to load, but also, when something is not found, it is automatically
> > searched.
> > 
> > In both options, the current module system has to be partly
> > rewritten, and some of the current drivers with module capability
> > might need to be modified to adapt to the new specifications.
> > 
> > And, a part from improving the current modular interface, there are
> > a lot of other devices that might benefit from it, not just the
> > block devices.
> > 
> > I still haven't looked at the memory footprint of QEMU, but I'm sure
> > that the QEMU binary will lose a lot of weight with this addition.
> 
> One think I don't see mentioned is how this impacts on QEMU feature
> detection by apps. For example, the recommended approach currnetly
> is to launch QEMU with 'qemu-system-BLAH --machine none
> -qmp /some/sock' and then query QMP for lists of devices supported,
> list of various backends and other features.
> 
> If you're going to suggest a fully modular system, then when doing
> QMP feature detection we still need to see the full list of features.
> So either that implies all the metadata associated with the modules
> remains built-in to QEMU, so QMP can answer this without lodaing the
> modules, or the QMP feature detection must imply auto-loading of all
> modules that exist.

Not everything can be trivially modularized.

But, I think that if we are able to get in modules the "very
independent" drivers, it will be already a huge improvement. And then,
we can think if it's worth the rest of the trouble.

Marc