Re: [Qemu-devel] Enablig DLPAR capacity on QEMU pSeries

[Erlon Cruz]

On Wed, Sep 12, 2012 at 12:53 PM, Alexander Graf <address@hidden> wrote:
> On 09/12/2012 04:54 PM, Erlon Cruz wrote:
>>
>> Hi all,
>>
>> We are planning to implement DLPAR capacity on QEMU pSeries. As we
>
>
> What is DLPAR? Hotplug support?

Yes, basically the way PowerVM uses to dynamically add memory, cpu,
and I/O slots to logical partitions (LPARs)

>
>> lack of experience in the internals of the arch we would like you guys
>> to give us some design directions
>> and confirm if we going in the right direction. Our first idea is:
>>
>>      1 - to patch 'spapr.c' so it can dynamically insert/remove basic
>> items into the device tree.
>
>
> What exactly would you like to patch into it? We already do have support for
> dynamic dt creation with the spapr target.
>

Actually we were not sure if the machine could do that. So we can add
things to the tree after booting it?

>>      2 - create a host side device that will be used with a guest side
>> driver to perform guest side operations and communicate changes from
>> host to the guest (like DynamicRM does in PowerVM LPARs). We are not
>
>
> Why not just use hypercalls?

The hypercalls are initiated from the guest side right? We also need a
way to the host send things to guest. Using hypercall also would
require a guest side KM.

>> planning to use powerpc-tools and want to make resource management
>> transparent (i.e. no need to run daemons or userspace programs in the
>> guest, only this kernel driver).
>>      3 - create bindings to support adding/removal  ibmvscsi devices
>>      4 - create bindings to support adding/removal  ibmveth devices
>>      5 - create bindings to support adding/removal PCI devices
>>      6 - create bindings to support adding/removal of memory
>
>
> This is going to be the hardest part. I don't think QEMU supports memory
> hotplug yet.

AFAIC ballonning is what QEMU provides so far which is fine to x86.

>
>>          - Do we need to do this the way PowerVM does? We have tested
>> virtio ballooning and it can works with a few endiannes corrections.
>
>
> I don't know how PowerVM works. But if normal ballooning is all you need,
> you should certainly just enable virtio-balloon.

PowerVM works with Logical Memory Blocks (LMB). The hypervisor
hotplugs memory blocks to guest's memory. Not only a 'borrowing' from
the guests, right Ben?

>>      7 - create bindings to support adding/removal  CPUs
>>          - is SMP supported already? I tried to run SMP in a x86 host
>> and the guest stuck when SMP is enabled
>
>
> SMP should work just fine, yes. Where exactly does it get stuck?

I think that is right after the guest kernel enables SMP

[    7.478259] Faulting instruction address: 0xc00000000053bbec
[    7.479521] Oops: Kernel access of bad area, sig: 11 [#1]
[    7.479694] SMP NR_CPUS=1024 NUMA pSeries

http://pastebin.com/VMtRyaTE

>
>
>>          - would be possible to work on this without a P7 baremetal
>> machine?
>
>
> At least for device hotplug, it should be perfectly possible to use an old
> G5 with PR KVM. I haven't gotten around to patch all the pieces of the
> puzzle to make -M pseries work with PR KVM when it's running on top of pHyp
> yet, so that won't work.
>
>
>> We have a P7 8205-E6B, is that possible to kick PHYP out?
>
>
> Ben?
>
>
>> Any ideia on how much effort (time/people) the hole thing would take?
>> Any consideration about this is much appreciated :)
>
>
> Phew. It's hard to tell. Depends heavily on how good your people are :).

Well, considering someone like you :p, so we would just need to
multiply it by 5 :P

>
>
> Alex
>
>