Re: [Qemu-devel] [RFC 0/3] Draft implementation of HPT resizing (qemu side)

[Alexander Graf]

> Am 29.01.2016 um 04:47 schrieb David Gibson <address@hidden>:
> 
>> On Thu, Jan 28, 2016 at 10:04:58PM +0100, Alexander Graf wrote:
>> 
>> 
>>> On 01/19/2016 12:02 PM, David Gibson wrote:
>>>> On Tue, Jan 19, 2016 at 01:18:17PM +0530, Bharata B Rao wrote:
>>>>> On Mon, Jan 18, 2016 at 04:44:38PM +1100, David Gibson wrote:
>>>>> Here is a draft qemu implementation of my proposed PAPR extension for
>>>>> allowing runtime resizing of a KVM/ppc64 guest's hash page table.
>>>>> That in turn will allow for more flexible memory hotplug.
>>>>> 
>>>>> This should work with the guest kernel side patches I also posted
>>>>> recently [1].
>>>>> 
>>>>> Still required to make this into a full implementation:
>>>>>  * Guest needs to auto-resize HPT on memory hotplug events
>>>>> 
>>>>>  * qemu needs to allocate HPT size based on current rather than
>>>>>    maximum memory if the guest is HPT resize aware
>>>>> 
>>>>>  * KVM host side implementation
>>>>> 
>>>>>  * PAPR standardization
>>>> So with the current patchset (QEMU and guest kernel changes), I should
>>>> be able to change the HTAB size of a PR guest right ? I see the below
>>>> failure though:
>>> Uh.. to be honest I haven't really considered the KVM case at all.
>>> I'm kind of surprised it didn't just refuse to do anything.
>>> 
>>>> address@hidden ~]# cat /sys/kernel/debug/powerpc/pft-size
>>>> 24
>>>> address@hidden ~]# echo 26 > /sys/kernel/debug/powerpc/pft-size
>>>> [   65.996845] lpar: Attempting to resize HPT to shift 26
>>>> [   65.996845] lpar: Attempting to resize HPT to shift 26
>>>> [   66.113596] lpar: HPT resize to shift 26 complete (109 ms / 6 ms)
>>>> [   66.113596] lpar: HPT resize to shift 26 complete (109 ms / 6 ms)
>>>> 
>>>> PR guest just hangs here while I see tons of below messages in
>>>> the 1st level guest:
>>>> 
>>>> KVM can't copy data from 0x3fff99e91400!
>>>> ...
>>>> Couldn't emulate instruction 0x00000000 (op 0 xop 0)
>>>> kvmppc_handle_exit_pr: emulation at 700 failed (00000000)
>>> Hm, not sure why that's happening.  At first I thought it was because
>>> we weren't updating SDR1 with the address of the new htab, but that's
>>> actually in there.  Maybe the KVM PR code isn't rereading it after
>>> initial VM startup.
>> 
>> The KVM PR code doesn't care - it just rereads SDR1 on every pteg lookup ;).
>> There's no caching at all.
> 
> Ok, no idea why it's not working then.  I'll investigate when I get a chance.
> 
>> Of course, the guest needs to invalidate all pending tlb entries if they're
>> now invalid.
>> 
>> Does this work on real hardware? Say, a G5?
> 
> As Paulus says it would be possible to do HPT resizing on real
> hardware, but the implementation I've done is specific to PAPR.  And
> obviously qemu wouldn't be relevant to that case.

So why make it specific to papr? Wouldn't it make sense to have it as a (ppc) 
generic interface in Linux?

For the PR PAPR case, QEMU allocates the HTAB, so it needs to make sure it 
pushes the changed address as new fake SDR1 value into kvm when it changes.


Alex