[Qemu-devel] Re: Should QMP be RPC to internal C interfaces?

[Markus Armbruster]

Anthony Liguori <address@hidden> writes:

> On 08/30/2010 09:52 AM, Markus Armbruster wrote:
>>> Because it's too easy to get it wrong in QEMU.  Here's the rationale.
>>>
>>> If I can't trivially call a QMP function in C, then I'm not going to
>>> use QMP functions within QEMU.  I'm not going to create an embedded
>>> JSON string just to call a function with three integer arguments.
>>>      
>> Yes, an internal interface is better done in idiomatic C, not with JSON
>> strings.
>>
>>    
>>> Yes, if we need to do that, we can create a C API that both the QMP
>>> interface uses and we also use internally but why?  All that does is
>>> introduce the chance that the C API will have more features than the
>>> QMP interface.
>>>      
>> Why is that bad?
>>
>> Internal and external interfaces have very different tradeoffs.
>>
>> An internal interface should eschew backward compatibility and embrace
>> change.
>>
>> An external interface needs to be stable, yet extensible.
>>    
>
> Nope.

Which part(s) do you disagree with?

(1) Internal and external interfaces have very different tradeoffs.

(2) An internal interface should eschew backward compatibility and
    embrace change.

(3) An external interface needs to be stable, yet extensible.

If none of the above, then how do you propose to resolve the tension
between external and internal interfaces?

> The internal interface should be the external interface.
>
> Otherwise, the external interface is going to rot.
>
>> It's therefore advisable to separate the two.  Otherwise the internal
>> interface gets bogged down with undue compatibility considerations
>> (backward&  forward), or the external interface suffers unnecessary
>> churn.
>>
>> When we designed QMP, we took special care to make it support compatible
>> evolution.  We consciously made it a proper protocol, not RPC to
>> internal C interfaces.  Are you proposing we go back to square one and
>> reargue the basics of QMP?
>>    
>
> All the pretty JSON interfaces don't matter if we're not exposing a
> useful API.
>
> We're trying to do too much.  We've been more or less completing
> ignoring the problem of creating a useful API for users to consume.
>
> We need to simplify.  We simplify by reducing scope.  Of the things
> that are important, a useful API is more important than whether it
> maps to your favorite dynamic language in an elegant way.

So you do propose we go back to square one and reargue the basics of
QMP.

>> No, the problem we suffer today is that we didn't design the external
>> interface properly above the level of basic protocol.  We took a
>> shortcut and converted existing monitor commands.  We've since
>> discovered we don't like that approach.
>>
>> Instead of giving up on protocol design without even trying and just
>> expose whatever internal interfaces strike us as useful via RPC, let's
>> design the external interface properly.
>>    
>
> What does that even mean?  How do you describe the external interface
> properly?
>
> It's a hell of a lot simpler to design the external interface as a C
> API, and then to implement the external interface as a C API.  Why
> make life harder than that?

You define the external interace the same how you define any interface:
you specify operations, arguments, returns and so forth.

C declarations are not a specification.  For instance, a C prototype is
only a part of a function's specification.  It says too little about
possible values, nothing about error conditions, semantics and so forth.

There are "a few" successful internet protocols that aren't RPC to a C
interface.

>>> I think it's a vitally important requirement that all future QMP
>>> functions have direct mappings to a C interface.
>>>      
>> Why?
>
> So that we can consume those APIs within QEMU.

QMP functions should consume the internal APIs.

A QMP function is concerned with interfacing and compatibility, and
leaves the actual work to the internal APIs.  Separation of concerns.

>>>                                                    The long term goal
>>> should be for that interface to be used by all of the command line
>>> arguments, SDL, and the human monitor.  If those things only relied on
>>> a single API and we exposed that API via QMP, than we would have an
>>> extremely useful interface.
>>>      
>> Yes, command line, human monitor and QMP should use internal interfaces
>> to do the real work, thus separate the real work neatly from
>> interface-specific stuff like parsing text.
>>
>> No, that doesn't mean we should expose internal interfaces via RPC.
>>    
>
> Having two interfaces guarantees failure.

The kernel tries hard to separate external and internal interfaces.
It's been failing quite successfully.

>                                            What's the separation
> between internal and external?  Is qdev internal or external?

qdev is the device model, i.e. a generic interface to devices.  qdev.h
is an internal interface.  It has a part facing generic code: the
generic interface to devices.  And it has a part facing devices: the
abstract interface the devices need to implement, plus stuff to help
with that.

The external interface is layered onto the internal interface.  -device
/ device_add expose a selected part of qdev externally.  info qtree
exposes some more.

We could do a better job specifying either of the interfaces.  But
removing the distinction between internal and external interface doesn't
improve the specification of the external interface one bit.

The internal interface is much richer.  It contains things that are of
no interest remotely, including but not limited to the part facing
devices.

It's designed for local, not remote use.  In particular, it makes
liberal use of pointers.  How would you handle those over RPC?

It can be changed easily, because the impact is limited to QEMU itself.
We don't do compatibility there, we just change it and fix up the users.