Re: [Qemu-devel] [RFC] QMP: add query-hotpluggable-cpus

[David Gibson]

On Tue, Feb 16, 2016 at 01:35:42PM +0100, Markus Armbruster wrote:
> Igor Mammedov <address@hidden> writes:
> 
> > On Mon, 15 Feb 2016 20:43:41 +0100
> > Markus Armbruster <address@hidden> wrote:
> >
> >> Igor Mammedov <address@hidden> writes:
> >> 
> >> > it will allow mgmt to query present and possible to hotplug CPUs
> >> > it is required from a target platform that wish to support
> >> > command to set board specific MachineClass.possible_cpus() hook,
> >> > which will return a list of possible CPUs with options
> >> > that would be needed for hotplugging possible CPUs.
> >> >
> >> > For RFC there are:
> >> >    'arch_id': 'int' - mandatory unique CPU number,
> >> >                       for x86 it's APIC ID for ARM it's MPIDR
> >> >    'type': 'str' - CPU object type for usage with device_add
> >> >
> >> > and a set of optional fields that would allows mgmt tools
> >> > to know at what granularity and where a new CPU could be
> >> > hotplugged;
> >> > [node],[socket],[core],[thread]
> >> > Hopefully that should cover needs for CPU hotplug porposes for
> >> > magor targets and we can extend structure in future adding
> >> > more fields if it will be needed.
> >> >
> >> > also for present CPUs there is a 'cpu_link' field which
> >> > would allow mgmt inspect whatever object/abstraction
> >> > the target platform considers as CPU object.
> >> >
> >> > For RFC purposes implements only for x86 target so far.  
> >> 
> >> Adding ad hoc queries as we go won't scale.  Could this be solved by a
> >> generic introspection interface?
> > Do you mean generic QOM introspection?
> 
> Possibly, but I don't want to prematurely limit the conversation to QOM
> introspection.
> 
> > Using QOM we could have '/cpus' container and create QOM links
> > for exiting (populated links) and possible (empty links) CPUs.
> > However in that case link's name will need have a special format
> > that will convey an information necessary for mgmt to hotplug
> > a CPU object, at least:
> >   - where: [node],[socket],[core],[thread] options
> >   - optionally what CPU object to use with device_add command
> 
> Encoding information in names feels wrong.

Yeah :(.

> > Another approach to do QOM introspection would be to model hierarchy 
> > of objects like node/socket/core..., That's what Andreas
> > worked on. Only it still suffers the same issue as above
> > wrt introspection and hotplug, One can pre-create empty
> > [nodes][sockets[cores]] containers at startup but then
> > leaf nodes that could be hotplugged would be a links anyway
> > and then again we need to give them special formatted names
> > (not well documented at that mgmt could make sense of).
> > That hierarchy would need to become stable ABI once
> > mgmt will start using it and QOM tree is quite unstable
> > now for that. For some targets it involves creating dummy
> > containers like node/socket/core for x86 where just modeling
> > a thread is sufficient.
> 
> I acknowledge your concern regarding QOM tree stability.  We have QOM
> introspection commands since 1.2.  They make the QOM tree part of the
> external interface, but we've never spelled out which parts of it (if
> any) are ABI.  Until we do, parts become de facto ABI by being used in
> anger.  As a result, we don't know something's ABI until it breaks.
> 
> Andreas, do you have an opinion on proper use of QOM by external
> software?
> 
> > The similar but a bit more abstract approach was suggested
> > by David https://lists.gnu.org/archive/html/qemu-ppc/2016-02/msg00000.html
> 
> Cc'ing him.

Actually I was already on the thread via my upstream email.

> If I understand the high-level idea correctly, David
> proposes to have an abstract type cpu-package with generic properties.
> Its concrete subtypes are composed of whatever components make up the
> hot-pluggable unit.

Yes, that's pretty much it.

> Management software can then use the generic properties to deal with hot
> plug without having to know about the concrete subtypes, at least to
> some useful degree.

That's the plan.

> Similarly, the generic properties suffice for implementing generic
> high-level interfaces like -smp.

Here it gets a bit fuzzier.  The idea is that the abstract type would
still make sense in a post -smp world allowing heterogenous setups.
However the concrete subtypes used for the time being are likely to
get their configuration from -smp, whether directly or indirectly.

My preferred option was for the machine type to "push" the smp
configuration into the package objects, rather than having them look
at the global directly.  However, in working with Bharata on a draft
implementation, I'm not actually sure how to do that.  So we might end
up looking at the global from the (concrete) package at least for the
time being.

At least, that's how it would work for semi-abstracted package types
as we have on some platforms.  For machine types which are supposed to
match real hardware closely, I'd could see package subtypes which are
hard-wired to have a particular number of threads/whatever based on
what the modelled device has.

> David, is that a fair summary?

Yes.

> Naturally, we need a way to introspect available subtypes of cpu-package
> to answer questions like what concrete types can actually be plugged
> into this board.

Actually, no.  Or at least, not yet.

The plan - as ameded by Eduardo's suggestion - is that in the first
cut the user can't directly construct cpu package objects.  For now
they all get constructed by the machine type (which already knows the
allowable types) and the user can just turn them on and off.

Later, we can allow more flexible user-directed construction of the
cpu packages.  That has a bunch more introspection details to thrash
out, but it shouldn't break the "easy case" option of having the
machine type pre-build the packages based on existing -smp and -cpu
options.

> This could be an instance of the generic QOM introspection question
> "what can plug into this socket"?  Unfortunately, I don't know enough
> QOM to put that into more concrete terms.  Andreas, Paolo, can you help
> out?
> 
> > Benefit of dedicated CPU hotplug focused QMP command is that
> > it can be quite abstract to suite most targets and not depend
> > on how a target models CPUs internally and still provide
> > information needed for hotplugging a CPU object.
> > That way we can split efforts on how we model/refactor CPUs
> > internally and how mgmt would work with them using
> > -device/device_add.
> 
> CPUs might be special enough to warrant special commands.  Nevertheless,
> non-special solutions should be at least explored.  That's what we're
> doing here.
> 

-- 
David Gibson                    | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
                                | _way_ _around_!
http://www.ozlabs.org/~dgibson

signature.asc
Description: PGP signature