Re: [Qemu-devel] [PATCH 0/3] cpu: numa: Fix the mapping initialization of VCPUs and NUMA nodes

[Igor Mammedov]

On Wed, 18 Jan 2017 20:40:04 +0800
Dou Liyang <address@hidden> wrote:

> As we fixed a bug(Bug 1) in below links, Named "Method-A":
> 
> https://lists.nongnu.org/archive/html/qemu-devel/2017-01/msg03354.html
> 
> Then, Eduardo gave us many suggests. Thanks very much!
> when we try them, we also find another bugs named "Bug 2".
I have an alternative fix for both issues for which
I've been writing cover letter when I saw this series.

My fix series more intrusive though as it's goal isn't
just to fix 'info numa' but rather switch away from
cpu-index based mapping to socket/core/thread based mapping
and stop using bitmaps for mapping. And as 'info numa' was
getting in the way, I fixed both issues in a slightly
different way.

So pls wait a bit, once travis build is completed,
I'll post series here.

> [Problem]
> ---------
> 
> As I use this command:
> 
> ./x86_64-softmmu/qemu-system-x86_64 \
>       -hda /image/fedora.img \
>       -m 1G,slots=4,maxmem=4G \
>       -enable-kvm \
>       -smp 2,maxcpus=16,sockets=4,cores=2,threads=2 \
>       -device qemu64-x86_64-cpu,id=cpu1,socket-id=0,core-id=1,thread-id=0 \
>       -device qemu64-x86_64-cpu,id=cpu2,socket-id=0,core-id=1,thread-id=1 \
>       -device qemu64-x86_64-cpu,id=cpu3,socket-id=1,core-id=0,thread-id=0 \
>       -device qemu64-x86_64-cpu,id=cpu4,socket-id=1,core-id=0,thread-id=1 \
>       -device qemu64-x86_64-cpu,id=cpu5,socket-id=1,core-id=1,thread-id=0 \
>       -device qemu64-x86_64-cpu,id=cpu6,socket-id=1,core-id=1,thread-id=1 \
>       -numa node,nodeid=0,cpus=0-3 \
>       -numa node,nodeid=1,cpus=4-7 \
>       -numa node,nodeid=2,cpus=8-11 \
>       -numa node,nodeid=3,cpus=12-15 \
>       -monitor stdio \
> 
> 1. Bug 1
> --------
> In Qemu monitor:
> 
> (qemu) info numa 
> 4 nodes
> node 0 cpus: 0 1 2 3 4 5 6 7
> node 0 size: 256 MB
> node 1 cpus:
> node 1 size: 256 MB
> node 2 cpus:
> node 2 size: 256 MB
> node 3 cpus:
> node 3 size: 256 MB
> 
> 2. Bug 2
> --------
> (qemu) device_add qemu64-x86_64-cpu,id=cpu7,socket-id=2,core-id=0,thread-id=0
> 
> (qemu) info numa
> 4 nodes
> node 0 cpus: 0 1 2 3 4 5 6 7 8
> node 0 size: 256 MB
> node 1 cpus:
> node 1 size: 256 MB
> node 2 cpus:
> node 2 size: 256 MB
> node 3 cpus:
> node 3 size: 256 MB
> 
> [Method-A]
> ----------
> 
> 1. Method-A that we provided above: 
>   * Ensure the numa_post_machine_init func in the appropriate location in
> vl.c::main().
> 
> It can fix Bug 1, but, can't work for Bug 2.
> 
> 1.1. For Bug 1: fixed
> 
> (qemu) info numa
> 4 nodes
> node 0 cpus: 0 1 2 3
> node 0 size: 256 MB
> node 1 cpus: 4 5 6 7
> node 1 size: 256 MB
> node 2 cpus:
> node 2 size: 256 MB
> node 3 cpus:
> node 3 size: 256 MB
> 
> 1.2. For Bug 2: can not fixed
> 
> (qemu) device_add qemu64-x86_64-cpu,id=cpu7,socket-id=2,core-id=0,thread-id=0
> 
> (qemu) info numa
> node 0 cpus: 0 1 2 3 8
> node 0 size: 256 MB
> node 1 cpus: 4 5 6 7
> node 1 size: 256 MB
> node 2 cpus:
> node 2 size: 256 MB
> node 3 cpus:
> node 3 size: 256 MB
> 
> [Solution]
> ----------
> 
> Move the CPUState::numa_node initialization to 
> qom/cpu.c:cpu_common_realizefn(),
> and remove numa_post_machine_init() completely.
> 
> It can fix Bug 1 and Bug 2. The result shows that:
> 
> (qemu) info numa
> 4 nodes
> node 0 cpus: 0 1 2 3
> node 0 size: 256 MB
> node 1 cpus: 4 5 6 7
> node 1 size: 256 MB
> node 2 cpus:
> node 2 size: 256 MB
> node 3 cpus:
> node 3 size: 256 MB
> 
> (qemu) device_add qemu64-x86_64-cpu,id=cpu7,socket-id=2,core-id=0,thread-id=0
> (qemu) info numa
> 4 nodes
> node 0 cpus: 0 1 2 3
> node 0 size: 256 MB
> node 1 cpus: 4 5 6 7
> node 1 size: 256 MB
> node 2 cpus: 8
> node 2 size: 256 MB
> node 3 cpus:
> node 3 size: 256 MB
> 
> (qemu) device_add qemu64-x86_64-cpu,id=cpu8,socket-id=3,core-id=0,thread-id=0
> (qemu) info numa
> 4 nodes
> node 0 cpus: 0 1 2 3
> node 0 size: 256 MB
> node 1 cpus: 4 5 6 7
> node 1 size: 256 MB
> node 2 cpus: 8
> node 2 size: 256 MB
> node 3 cpus: 12
> node 3 size: 256 MB
> 
> (qemu) device_del cpu5
> (qemu) info numa
> 4 nodes
> node 0 cpus: 0 1 2 3
> node 0 size: 256 MB
> node 1 cpus: 4 5 7
> node 1 size: 256 MB
> node 2 cpus: 8
> node 2 size: 256 MB
> node 3 cpus: 12
> node 3 size: 256 MB
> 
> Dou Liyang (3):
>   cpu: Make the mapping of CPUs and NUMA nodes in cpu_common_realizefn
>   numa: Remove the numa_post_machine_init function
>   cpu: make the function of cpu_common_map_numa_node more efficiently
> 
>  include/sysemu/numa.h |  1 -
>  numa.c                | 15 ---------------
>  qom/cpu.c             | 16 ++++++++++++++++
>  vl.c                  |  2 --
>  4 files changed, 16 insertions(+), 18 deletions(-)
>