Re: [Qemu-devel] [RFC] QMP: add query-hotpluggable-cpus

[Markus Armbruster]

David Gibson <address@hidden> writes:

> On Tue, Feb 16, 2016 at 01:35:42PM +0100, Markus Armbruster wrote:
>> Igor Mammedov <address@hidden> writes:
>> 
>> > On Mon, 15 Feb 2016 20:43:41 +0100
>> > Markus Armbruster <address@hidden> wrote:
>> >
>> >> Igor Mammedov <address@hidden> writes:
>> >> 
>> >> > it will allow mgmt to query present and possible to hotplug CPUs
>> >> > it is required from a target platform that wish to support
>> >> > command to set board specific MachineClass.possible_cpus() hook,
>> >> > which will return a list of possible CPUs with options
>> >> > that would be needed for hotplugging possible CPUs.
>> >> >
>> >> > For RFC there are:
>> >> >    'arch_id': 'int' - mandatory unique CPU number,
>> >> >                       for x86 it's APIC ID for ARM it's MPIDR
>> >> >    'type': 'str' - CPU object type for usage with device_add
>> >> >
>> >> > and a set of optional fields that would allows mgmt tools
>> >> > to know at what granularity and where a new CPU could be
>> >> > hotplugged;
>> >> > [node],[socket],[core],[thread]
>> >> > Hopefully that should cover needs for CPU hotplug porposes for
>> >> > magor targets and we can extend structure in future adding
>> >> > more fields if it will be needed.
>> >> >
>> >> > also for present CPUs there is a 'cpu_link' field which
>> >> > would allow mgmt inspect whatever object/abstraction
>> >> > the target platform considers as CPU object.
>> >> >
>> >> > For RFC purposes implements only for x86 target so far.  
>> >> 
>> >> Adding ad hoc queries as we go won't scale.  Could this be solved by a
>> >> generic introspection interface?
>> > Do you mean generic QOM introspection?
>> 
>> Possibly, but I don't want to prematurely limit the conversation to QOM
>> introspection.
>> 
>> > Using QOM we could have '/cpus' container and create QOM links
>> > for exiting (populated links) and possible (empty links) CPUs.
>> > However in that case link's name will need have a special format
>> > that will convey an information necessary for mgmt to hotplug
>> > a CPU object, at least:
>> >   - where: [node],[socket],[core],[thread] options
>> >   - optionally what CPU object to use with device_add command
>> 
>> Encoding information in names feels wrong.
>
> Yeah :(.
>
>> > Another approach to do QOM introspection would be to model hierarchy 
>> > of objects like node/socket/core..., That's what Andreas
>> > worked on. Only it still suffers the same issue as above
>> > wrt introspection and hotplug, One can pre-create empty
>> > [nodes][sockets[cores]] containers at startup but then
>> > leaf nodes that could be hotplugged would be a links anyway
>> > and then again we need to give them special formatted names
>> > (not well documented at that mgmt could make sense of).
>> > That hierarchy would need to become stable ABI once
>> > mgmt will start using it and QOM tree is quite unstable
>> > now for that. For some targets it involves creating dummy
>> > containers like node/socket/core for x86 where just modeling
>> > a thread is sufficient.
>> 
>> I acknowledge your concern regarding QOM tree stability.  We have QOM
>> introspection commands since 1.2.  They make the QOM tree part of the
>> external interface, but we've never spelled out which parts of it (if
>> any) are ABI.  Until we do, parts become de facto ABI by being used in
>> anger.  As a result, we don't know something's ABI until it breaks.
>> 
>> Andreas, do you have an opinion on proper use of QOM by external
>> software?
>> 
>> > The similar but a bit more abstract approach was suggested
>> > by David https://lists.gnu.org/archive/html/qemu-ppc/2016-02/msg00000.html
>> 
>> Cc'ing him.
>
> Actually I was already on the thread via my upstream email.
>
>> If I understand the high-level idea correctly, David
>> proposes to have an abstract type cpu-package with generic properties.
>> Its concrete subtypes are composed of whatever components make up the
>> hot-pluggable unit.
>
> Yes, that's pretty much it.
>
>> Management software can then use the generic properties to deal with hot
>> plug without having to know about the concrete subtypes, at least to
>> some useful degree.
>
> That's the plan.
>
>> Similarly, the generic properties suffice for implementing generic
>> high-level interfaces like -smp.
>
> Here it gets a bit fuzzier.  The idea is that the abstract type would
> still make sense in a post -smp world allowing heterogenous setups.
> However the concrete subtypes used for the time being are likely to
> get their configuration from -smp, whether directly or indirectly.
>
> My preferred option was for the machine type to "push" the smp
> configuration into the package objects, rather than having them look
> at the global directly.  However, in working with Bharata on a draft
> implementation, I'm not actually sure how to do that.  So we might end
> up looking at the global from the (concrete) package at least for the
> time being.
>
> At least, that's how it would work for semi-abstracted package types
> as we have on some platforms.  For machine types which are supposed to
> match real hardware closely, I'd could see package subtypes which are
> hard-wired to have a particular number of threads/whatever based on
> what the modelled device has.

A machine type that models a board without cold-pluggable CPU slots
would have a single tuple of CPU packages.

A machine type that models a board with a fixed number of pluggable
slots would have a fixed set of tuples to choose from.

Other machine types may have infinite sets, limited only by resource
constraints in practice.

>> David, is that a fair summary?
>
> Yes.
>
>> Naturally, we need a way to introspect available subtypes of cpu-package
>> to answer questions like what concrete types can actually be plugged
>> into this board.
>
> Actually, no.  Or at least, not yet.
>
> The plan - as ameded by Eduardo's suggestion - is that in the first
> cut the user can't directly construct cpu package objects.  For now
> they all get constructed by the machine type (which already knows the
> allowable types) and the user can just turn them on and off.
>
> Later, we can allow more flexible user-directed construction of the
> cpu packages.  That has a bunch more introspection details to thrash
> out, but it shouldn't break the "easy case" option of having the
> machine type pre-build the packages based on existing -smp and -cpu
> options.

Makes sense to me.  Also avoids having to boil the QOM introspection
pond first.

>> This could be an instance of the generic QOM introspection question
>> "what can plug into this socket"?  Unfortunately, I don't know enough
>> QOM to put that into more concrete terms.  Andreas, Paolo, can you help
>> out?
>> 
>> > Benefit of dedicated CPU hotplug focused QMP command is that
>> > it can be quite abstract to suite most targets and not depend
>> > on how a target models CPUs internally and still provide
>> > information needed for hotplugging a CPU object.
>> > That way we can split efforts on how we model/refactor CPUs
>> > internally and how mgmt would work with them using
>> > -device/device_add.
>> 
>> CPUs might be special enough to warrant special commands.  Nevertheless,
>> non-special solutions should be at least explored.  That's what we're
>> doing here.

Thanks!