Re: [Qemu-devel] [RFC] QCFG: a new mechanism to replace QemuOpts and option handling

[Markus Armbruster]

Anthony Liguori <address@hidden> writes:

> On 03/22/2011 08:01 AM, Markus Armbruster wrote:
>> Type checking macros are feasible (see [*] for an existence proof), but
>> things do get hairy, and the resulting error messages can be less than
>> clear at times.
>
> That just gives you a warning.  You can do much better things with
> __builtin_types_compatible_p.

Quote of existence proof != this is the best way to do it :)

> But neither really solves the problem I'm talking about.  I can go
> into it in depth but hopefully we both can agree that trying to build
> introspection macros is pretty difficult if not impossible :-)

Depends on what you need from them.

>>>                                                              However,
>>> this makes it very difficult to support things like lists of lists or
>>> anything else that would basically require a non-concrete type.
>> Sounds like you want a more expressive type system than C's, and propose
>> to get it by building your own DSL.  I'm not sure that's wise.
>
> It's an IDL.  IDL based RPCs are pretty common with C.  The IDL is
> purely declarative.

Still sounds like you want a more expressive type system than C's, and
propose to build it ourselves.

First step would be to define the type system rigorously.

>>> If you plan to expose these types in a library, you need to either
>>> explicitly pad each structure and make sure that the padding is
>>> updated correctly each time a new member is added.
>> As long as the data description is data, writing a program to check that
>> a new version is compatible with the old one shouldn't be hard.
>
> If you define the structs on your own, you need to either have a data
> description of the padding or be very careful doing it yourself.

With a program to check compatibility, you can easily make the build
fail when you break compatibility.

>>>                                                      Alternatively, you
>>> can add an allocation function that automatically pads each structure
>>> transparently.
>> Weaker than a comprehensive check, but could be good enough.
>>
>>> qmp-gen.py creates qmp-types.[ch] to do exactly the above and also
>>> generates the type declaration so that you don't have to duplicate the
>>> type marshalling code and the type declaration.  Today, this is close
>>> to 2k LOCs so it's actually a significant amount of code code.
>>>
>>> There is also the code that takes the input (via QCFG or QMP) and
>>> calls an appropriate C function with a strongly typed argument.  I've
>> Not sure I got you here.  Perhaps an example could enlighten me :)
>
> void qapi_free_vnc_info(VncInfo *obj)
> {
>     if (!obj) {
>         return;
>     }
>     if (obj->has_host) {
>         qemu_free(obj->host);
>     }
>     if (obj->has_family) {
>         qemu_free(obj->family);
>     }
>     if (obj->has_service) {
>         qemu_free(obj->service);
>     }
>     if (obj->has_auth) {
>         qemu_free(obj->auth);
>     }
>     if (obj->has_clients) {
>         qapi_free_vnc_client_info(obj->clients);
>     }
>
>
>     qapi_free_vnc_info(obj->next);
>     qemu_free(obj);
> }
>
> It's pretty basic boiler plate code that could be written by hand, but
> why not generate it.  It actually all adds up pretty quickly in terms
> of SLOCs.

Since we have the data description anyway, why not

    for all members
        if member needs destruction
            destroy it

Just because we can generate boiler-plate code doesn't mean we should
have boiler-plate code.

>>> The mechanism I described using the visitor pattern is really the
>>> right solution for vmstate.  The hard problems to solve for vmstate
>>> are:
>>>
>>> 1) How to we support old versions in a robust way.  There are fancy
>>> things we could do once we have a proper visitor mechanism.  We could
>>> have special marshallers for old versions, we could treat the output
>>> of the visitor as an in memory tree and do XSLT style translations,
>>> etc.
>>>
>>> 2) How do we generate the visitor for each device.  I don't think it's
>>> practical to describe devices in JSON.  It certainly simplifies the
>>> problem but it seems ugly to me.  I think we realistically need a C
>>> style IDL and adopt a style of treating it as a header.
>> Now I'm confused.  Do you mean your JSON-based DSL won't cut it for
>> vmstate?
>>
>> If yes, why is it wise to go down that route now?
>
> There are a few paths we could go.  We can describe devices in JSON.
> This makes VMState introspectable with all of the nice properties of
> everything else.  But the question is, do we describe the full device
> state and then use a separate mechanism to cull out the bits that can
> be recomputed.
>
> To we only describe the guest visible state and treat that as a
> separate structure?  Is that embedded in the main state object or do
> we explicitly translate the main state object to this new type?

Requiring fields of the externally visible vmstate wire format to be
backed by members of the device state struct (whether they are right in
the outermost struct or not) creates a potentially troublesome tight
coupling between wire format and internal representation.

You can avoid such coupling by permitting wire format fields to be
backed by functions rather than struct members.

> We can pretty easily have a C-like IDL so I'm not terribly concerned
> about describing devices in JSON.  It's about finding the right way to
> structure device marshalling in the long term.
>
> So yes, I think JSON is the right approach, but that doesn't mean I've
> figured out how to do VMState.
>
>>> Yeah, this is one of the big challenges with vmstate.  There needs to
>>> be a clearer line between internal types and object models vs. the
>>> externally visible interfaces.
>> Not only with vmstate.  If we couple QMP closely to internal interfaces,
>> I'm afraid we'll end up in the same unhappy place we're now with
>> vmstate.
>
> Yeah, it's a tough balance to strike.  If you expose too much detail
> about internals, it's very difficult to maintain compatibility in the
> long term.  If you have a pure external interface, it's difficult to
> make sure that the external interfaces are actually useful because
> there's no immediate feedback mechanism.

The only valid judges on the usefulness of external interfaces are
external users.

Coupling them tightly to internal interfaces just to shortcut that
judgement doesn't strike me as a smart idea.  Please note the "tightly"
there.

>
> Regards,
>
> Anthony Liguori
>
>>> Regards,
>>>
>>> Anthony Liguori
>>>
>>>> [...]
>>>>
>> [*] http://ccan.ozlabs.org/info/check_type.html