qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [PATCH RFC 0/4] target-i386: PC socket/core/thread mode


From: Christian Borntraeger
Subject: Re: [Qemu-devel] [PATCH RFC 0/4] target-i386: PC socket/core/thread modeling, part 1
Date: Tue, 24 Mar 2015 15:33:47 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0

Am 23.03.2015 um 18:31 schrieb Andreas Färber:
> Hello,
> 
> This long-postponed series proposes a hierarchical QOM model of socket
> and core objects for the x86 PC machines.

Just some comments from the s390 side as we probably want more than status 
quo in the future as well.

Traditionally all CPUs were equal from the software side, but hardware was
already organized in a node like fashion.

Starting with z10 (2008) the cpu topology was exposed to software (not the
memory topology!) which allowed Linux to optimize scheduling based on
locality.
So on an LPAR you might see something like
$ lscpu -e
CPU BOOK SOCKET CORE L1d:L1i:L2 ONLINE CONFIGURED POLARIZATION ADDRESS
0   0    0      0    0:0:0      ja     ja         horizontal   0
1   0    0      1    1:1:1      ja     ja         horizontal   1
2   0    0      2    2:2:2      ja     ja         horizontal   2
3   0    1      3    3:3:3      ja     ja         horizontal   3
4   0    1      4    4:4:4      ja     ja         horizontal   4
5   0    1      5    5:5:5      ja     ja         horizontal   5
6   0    2      6    6:6:6      ja     ja         horizontal   6
7   0    2      7    7:7:7      ja     ja         horizontal   7
8   0    2      8    8:8:8      ja     ja         horizontal   8
9   0    3      9    9:9:9      ja     ja         horizontal   9
10  0    3      10   10:10:10   ja     ja         horizontal   10
11  0    3      11   11:11:11   ja     ja         horizontal   11
12  1    4      12   12:12:12   ja     ja         horizontal   12
13  1    4      13   13:13:13   ja     ja         horizontal   13
14  1    4      14   14:14:14   ja     ja         horizontal   14
15  1    5      15   15:15:15   ja     ja         horizontal   15


Under z/VM the topology is flat (each CPU on a separate node)
# lscpu -e
CPU BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS
0   0    0      0    0:0:0:0         ja     ja         horizontal   0
1   1    1      1    1:1:1:1         ja     ja         horizontal   1
2   2    2      2    2:2:2:2         ja     ja         horizontal   2

Thats what we have today in KVM as well. Now...
> 
> Background is that due to qdev limitations we had to introduce an ICC bus
> to be able to hot-add CPUs and their APICs. By now this limitation could be
> resolved via a QOM hotplug handler interface.
> 
> However, the QOM hotplug model is associated with having link<> properties.
> Given that physically we cannot hot-add hyperthreads but full CPU sockets,
> this series prepares the underbelly for having those link properties be of
> the new type X86CPUSocket rather than X86CPU.
> 
> As final step in this series, the X86CPU allocation model is changed to be
> init'ed in-place, as part of an "atomic" socket object. A follow-up will be
> to attempt the same in-place allocation model for the APIC; one difficulty
> there is that several places do a NULL check for the APIC pointer as quick
> way of telling whether an APIC is present or not.
> 
> NOT IN THIS SERIES is converting cpu-add to the same socket/core/thread model
> and initializing them in-place. The current PoC implementation assumes that
> CPUs get added sequentially and that the preceding CPU can be used to obtain
> the core via unclean QOM parent accesses.
> IIUC that must be changed so that an arbitrary thread added via cpu-add
> creates the full socket and core(s). That would work best if indexed link<>
> properties could be used. That's an undecided design question of whether
> we want to have /machine/cpu-socket[n] or whether it makes sense to integrate
> NUMA modeling while at it and rather have /machine/node[n]/socket[m].

...looking into the future, we probably want to be able to model a cpu node
topology as newer machines are getting bigger and bigger and the locality
gets more important. So maybe we want to have a KVM-guest with 40 vCPUs and
pin 20 vCPUs on host book (node) 0 and 20 vcpu on host book 1.
- so node is of interest.


Regarding threads: z13 has SMT2. Right now with upstream code, we can fan 
out at the lpar level so that each thread becomes a host CPU and that is
used to back guest vCPUs. The alternative to pass a full host core to 
the guest and do the fan out in the guest is currently not supported.

Regarding cpu hotplug: This work is currently halted due to other things
but we plan to continue to work on that. 

I will have a look at your code. Thanks for working on this.

> 
> Note that this socket modeling is not only PC-specific in the softmmu sense
> but also in that not every X86CPU must be on a socket (e.g., Quark X1000).
> Therefore it was a concious decision to not label some things target-i386
> and to place code in pc.c rather than cpu.c.
> 
> Further note that it is ignored here that -smp enforces that AMD CPUs don't
> have hyperthreads, i.e. AMD X86CPUs will have only one thread[n] child<>.
> 
> Context:
> 
>    qemu.git master
>    "pc: Ensure non-zero CPU ref count after attaching to ICC bus"
> -> this series adding socket/core objects
>    cpu-add conversion
>    APIC cleanups
> 
> Available for testing here:
> git://github.com/afaerber/qemu-cpu.git qom-cpu-x86-sockets-1.v1
> https://github.com/afaerber/qemu-cpu/commits/qom-cpu-x86-sockets-1.v1
> 
> Regards,
> Andreas
> 
> Cc: Eduardo Habkost <address@hidden>
> Cc: Igor Mammedov <address@hidden>
> Cc: Paolo Bonzini <address@hidden>
> Cc: Peter Maydell <address@hidden>
> Cc: Bharata B Rao <address@hidden>
> Cc: Christian Borntraeger <address@hidden>
> 
> Andreas Färber (4):
>   cpu: Prepare Socket container type
>   target-i386: Prepare CPU socket/core abstraction
>   pc: Create sockets and cores for CPUs
>   pc: Create initial CPUs in-place
> 
>  hw/cpu/Makefile.objs         |  2 +-
>  hw/cpu/socket.c              | 21 ++++++++++
>  hw/i386/Makefile.objs        |  1 +
>  hw/i386/cpu-core.c           | 45 +++++++++++++++++++++
>  hw/i386/cpu-socket.c         | 45 +++++++++++++++++++++
>  hw/i386/pc.c                 | 95 
> ++++++++++++++++++++++++++++++++++++++++----
>  include/hw/cpu/socket.h      | 14 +++++++
>  include/hw/i386/cpu-core.h   | 29 ++++++++++++++
>  include/hw/i386/cpu-socket.h | 29 ++++++++++++++
>  9 files changed, 272 insertions(+), 9 deletions(-)
>  create mode 100644 hw/cpu/socket.c
>  create mode 100644 hw/i386/cpu-core.c
>  create mode 100644 hw/i386/cpu-socket.c
>  create mode 100644 include/hw/cpu/socket.h
>  create mode 100644 include/hw/i386/cpu-core.h
>  create mode 100644 include/hw/i386/cpu-socket.h
> 




reply via email to

[Prev in Thread] Current Thread [Next in Thread]