Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Zhu Guihua]

On Mon, 2015-01-26 at 04:25 +0100, Alexandre DERUMIER wrote:
> >>2)when numa is used, the hotplugged cpu is always on numa node 0 
> >>(cpu_add or device_add cpu) 
> 
> About this, it seem to be a display bug in "info numa",
> cpu is correctly assigned to numa node1 in the guest.
> 
> 

Now in the guest, VCPUs are assigned round-robin.
But in qemu, vcpus' property 'numa_node' was assigned the default
value 0.

We have made a patchset to solve this successfully.
But we plan to send the patchset to the community later.

Regards,
Zhu

> ----- Mail original -----
> De: "aderumier" <address@hidden>
> À: "Zhu Guihua" <address@hidden>
> Cc: "qemu-devel" <address@hidden>, address@hidden, "guz fnst" 
> <address@hidden>, "isimatu yasuaki" <address@hidden>, "Anshul Makkar" 
> <address@hidden>, "chen fan fnst" <address@hidden>, "Igor Mammedov" 
> <address@hidden>, "afaerber" <address@hidden>
> Envoyé: Lundi 26 Janvier 2015 04:19:52
> Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support
> 
> Thanks for your reply. 
> 
> 2 others things: 
> 
> 1) 
> on cpu unplug, I see that the cpu is correctly removed from my linux guest 
> but not from qemu 
> 
> starting with a guest with 3cpus: 
> 
> guest: #ls -lah /sys/devices/system/ |grep cpu 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu2 
> 
> hmp: # info cpus 
> * CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 
> CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 
> CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102 
> 
> 
> then unplug cpu2 
> hmp : device_del cpu2 
> 
> guest: 
> 
> dmesg: 
> [ 176.219754] Unregister pv shared memory for cpu 2 
> [ 176.278881] smpboot: CPU 2 is now offline 
> 
> #ls -lah /sys/devices/system/ |grep cpu 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1 
> 
> hmp: # info cpus 
> * CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 
> CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 
> CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102 
> 
> 
> 
> 
> 2)when numa is used, the hotplugged cpu is always on numa node 0 
> (cpu_add or device_add cpu) 
> 
> 
> starting a guest, with 2 sockets,1 cores 
> 
> -smp 2,sockets=2,cores=1,maxcpus=2 
> -object memory-backend-ram,size=256M,id=ram-node0 -numa 
> node,nodeid=0,cpus=0,memdev=ram-node0 
> -object memory-backend-ram,size=256M,id=ram-node1 -numa 
> node,nodeid=1,cpus=1,memdev=ram-node1 
> 
> hmp: 
> # info numa 
> 2 nodes 
> node 0 cpus: 0 
> node 0 size: 256 MB 
> node 1 cpus: 1 
> node 1 size: 256 MB 
> 
> ok 
> 
> now 
> 
> starting with same topology, but with 1cpu at start 
> -smp 2,sockets=2,cores=1,maxcpus=2 
> -object memory-backend-ram,size=256M,id=ram-node0 -numa 
> node,nodeid=0,cpus=0,memdev=ram-node0 
> -object memory-backend-ram,size=256M,id=ram-node1 -numa 
> node,nodeid=1,cpus=1,memdev=ram-node1 
> 
> # info numa 
> 2 nodes 
> node 0 cpus: 0 
> node 0 size: 256 MB 
> node 1 cpus: 
> node 1 size: 256 MB 
> 
> hotpluging a cpu 
> # device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
> 
> # info numa 
> 2 nodes 
> node 0 cpus: 0 1 
> node 0 size: 256 MB 
> node 1 cpus: 
> node 1 size: 256 MB 
> 
> cpu1 should be on node1, not node0. 
> 
> 
> Regards, 
> 
> Alexandre 
> 
> ----- Mail original ----- 
> De: "Zhu Guihua" <address@hidden> 
> À: "aderumier" <address@hidden> 
> Cc: "qemu-devel" <address@hidden>, address@hidden, "guz fnst" 
> <address@hidden>, "isimatu yasuaki" <address@hidden>, "Anshul Makkar" 
> <address@hidden>, "chen fan fnst" <address@hidden>, "Igor Mammedov" 
> <address@hidden>, "afaerber" <address@hidden> 
> Envoyé: Lundi 26 Janvier 2015 03:01:48 
> Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support 
> 
> On Fri, 2015-01-23 at 11:24 +0100, Alexandre DERUMIER wrote: 
> > Hello, 
> > 
> > I'm currently testing the new cpu unplug features, 
> > Works fine here with debian guests and kernel 3.14. 
> > 
> 
> Thanks for your test. 
> 
> > But I have notice some small potential bugs, but I'm not sure I'm doing it 
> > right. 
> > 
> > 1)first, to unplug cpu, we need an id for cpu 
> > 
> 
> Yes, if you want to unplug cpu, you must have an id for cpu. 
> 
> > The problem is that the current qemu command line 
> > -smp 1,sockets=2,cores=1,maxcpus=2 
> > 
> > for example, will create 1 cpu on apic-id 0 without any id, so we can't 
> > unplug it. 
> > 
> > 
> > So, I have tried with 
> > 
> > -smp 1,sockets=2,cores=1,maxcpus=2 -device 
> > kvm64-x86_64-cpu,apic-id=0,id=cpu0 
> > 
> > But this give me an error: 
> > "-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists" 
> > 
> 
> APIC ID 0 was used by the cpu of '-smp 1'. 
> So you should use apic-id=1 
> 
> > (also try to set -smp 0, but it's not working). 
> > 
> > 
> > 
> > 2) second problem, if I start with 
> > -smp 1,sockets=2,cores=1,maxcpus=2 
> > 
> > then hmp: 
> > device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
> > 
> > then hmp : device_del cpu1 
> > 
> > Got an error:" 
> > This is the last cpu, should not be removed!" 
> > 
> > 
> 
> Oh, it's our problem, thanks for your pointing out. 
> I will fix it in next version. 
> 
> Regards, 
> Zhu 
> 
> > 
> > This is coming from 
> > [PATCH 06/12] pc: add cpu hot unplug request callback support 
> > + if (smp_cpus == 1) { 
> > + error_setg(&local_err, 
> > + "This is the last cpu, should not be removed!"); 
> > + goto out; 
> > + } 
> > 
> > 
> > 
> > So, the only way unplug is working for me, is to start with -smp 2 minimum 
> > -smp 2,sockets=2,cores=1,maxcpus=4 
> > 
> > Then I can hotplug|unplug cpuid >= 2 
> > 
> > 
> > 
> > Regards, 
> > 
> > Alexandre Derumier 
> [...] 
>