Re: [Qemu-ppc] CPU hotplug

[David Gibson]

On Wed, Feb 03, 2016 at 10:33:48AM +0530, Bharata B Rao wrote:
> On Mon, Feb 01, 2016 at 04:35:17PM +1100, David Gibson wrote:
> > Hi,
> > 
> > It seems to me we're getting rather bogged down in how to proceed with
> > an improved CPU hotplug (and hot unplug) interface, both generically
> > and for ppc in particular.
> > 
> > So here's a somewhat more concrete suggestion of a way forward, to see
> > if we can get some consensus.
> > 
> > The biggest difficulty I think we're grappling with is that device-add
> > is actually *not* a great interface to cpu hotplug.  Or rather, it's
> > not great as the _only_ interface: in order to represent the many
> > different constraints on how cpus can be plugged on various platforms,
> > it's natural to use a heirarchy of cpu core / socket / package types
> > specific to the specific platform or real-world cpu package being
> > modeled.  However, for the normal case of a regular homogenous (and at
> > least slightly para-virtualized) server, that interface is nasty for
> > management layers because they have to know the right type to
> > instantiate.
> > 
> > To address this, I'm proposing this two layer interface:
> > 
> > Layer 1: Low-level, device-add based
> > 
> >     * a new, generic cpu-package QOM type represents a group of 1 or
> >       more cpu threads which can be hotplugged as a unit
> >     * cpu-package is abstract and can't be instantiated directly
> >     * archs and/or individual platforms have specific subtypes of
> >       cpu-package which can be instantiated
> >     * for platforms attempting to be faithful representations of real
> >       hardware these subtypes would match the specific characteristics
> >       of the real hardware devices.  In addition to the cpu threads,
> >       they may have other on chip devices as sub-objects.
> >     * for platforms which are paravirtual - or which have existing
> >       firmware abstractions for cpu cores/sockets/packages/whatever -
> >       these could be more abstract, but would still be tied to that
> >       platform's constraints
> >     * Depending on the platform the cpu-package object could have
> >       further internal structure (e.g. a package object representing a
> >       socket contains package objects representing each core, which in
> >       turn contain cpu objects for each thread)
> >         * Some crazy platform that has multiple daughterboards each with
> >           several multi-chip-modules each with several chips, each
> >       with several cores each with several threads could represent
> >       that too.
> > 
> > What would be common to all the cpu-package subtypes is:
> >     * A boolean "present" attribute ("realized" might already be
> >       suitable, but I'm not certain)
> >     * A generic means of determining the number of cpu threads in the
> >       package, and enumerating those
> >     * A generic means of determining if the package is hotpluggable or
> >       not
> >     * They'd get listed in a standard place in the QOM tree
> > 
> > This interface is suitable if you want complete control over
> > constructing the system, including weird cases like heterogeneous
> > machines (either totally different cpu types, or just different
> > numbers of threads in different packages).
> > 
> > The intention is that these objects would never look at the global cpu
> > type or sockets/cores/threads numbers.  The next level up would
> > instead configure the packages to match those for the common case.
> > 
> > Layer 2: Higher-level
> > 
> >     * not all machine types need support this model, but I'd expect
> >       all future versions of machine types designed for production use
> >       to do so
> >     * machine types don't construct cpu objects directly
> >     * instead they create enough cpu-package objects - of a subtype
> >       suitable for this machine - to provide maxcpus threads
> >     * the machine type would set the "present" bit on enough of the
> >       cpu packages to provide the base number of cpu threads
> 
> In the generic cpu-core RFC that I posted last year
> (https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg01526.html),
> I did have backend objects (which I called them sockets) into which
> the generic cpu-core device would fit it and I used the QOM links to
> bring out the notion of cpu-core device populating the socket.
> 
> I had the sockets as backend objects and created as many of them as needed
> upfront to fit the max_cpus. These objects weren't exposed them to the user,
> but instead the cpu-core device was exposed to the user.

Right, as I mentioned on IRC this is based partly on your earlier
proposal.

The big difference, as I see it, is that in this proposal the cpu
package objects aren't linked directly to the socket/core/thread
heirarchy - different platforms can place them differently based on
what works for them.

> However, I like the current proposal where Layer 2 interface is exposed to the
> user and letting archs build up the CPU topology underneath in the manner
> that they deem fit for the arch.
> 
> Regards,
> Bharata.
> 

-- 
David Gibson                    | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
                                | _way_ _around_!
http://www.ozlabs.org/~dgibson

signature.asc
Description: PGP signature