Re: [Qemu-devel] [PATCH] ppc/pnv: Add model for Power8 PHB3 PCIe Host bridge

[Cédric Le Goater]

On 06/27/2018 02:18 PM, Cédric Le Goater wrote:
> On 06/27/2018 12:22 PM, Andrea Bolognani wrote:
>> On Tue, 2018-06-26 at 19:02 +0200, Cédric Le Goater wrote:
>>> On 06/26/2018 05:57 PM, Andrea Bolognani wrote:
>>>> On Tue, 2018-06-26 at 15:59 +0200, Cédric Le Goater wrote:
>>>>> This is a model of the PCIe host bridge found on Power8 chips,
>>>>> including PowerBus logic interface, IOMMU support, PCIe root complex,
>>>>> XICS MSI and LSI interrupt sources.
>>>>>
>>>>> 4 PHBs are provisioned under the Power8 chip model to fit hardware but
>>>>> only one is currently initialized.
>>>>
>>>> What's the advantage in creating 4 PHBs instead of a single one,
>>>
>>> The Power8 chip comes in different flavors: Venice, Murano, Naple, 
>>> each having a different number of PHBs. We don't need to initialize 
>>> them all to plug only a couple of devices (net, storage, usbs) 
>>>
>>> When time comes, we might want to test some more complex configurations
>>> or extend the modeling with CAPI support. That's why we have a :
>>>
>>>     #define PNV_MAX_CHIP_PHB 4
>>>         PnvPHB3      phbs[PNV_MAX_CHIP_PHB];
>>>
>>> under the chip, and a 'num_phbs' attribute to increase the number
>>> of controllers. It still needs to be tested but that's the goal.
>>
>> I was a bit confused about the difference between "provisioning"
>> and "initializing" a PHB, I guess.
> 
> objects have different states : allocated, initialized, realized
> I guess "provision (memory)" is not the best choice for the first
> state.
> 
>> Now that I've looked at the code, my (very uneducated) reading is
>> that you're allocating memory for 4 PHBs along with the chip, but
>> only actually initializing num_phbs of them, with 1 being the
>> default.
> 
> Yes. I had the idea to increase the number of PHBs with a machine
> option later on if needed.
> 
>> I have no idea what this implies when it comes to adding PCI
>> controller to the guest, though. If I run a guest with num_phbs=1,
>> are ids pci.1..pci.3 reserved even though the corresponding PHBs
>> have not been initialized? 
> 
> They don't exist on the PowerNV machine if you don't have a PHB3
> device backing them.
> 
>> Is num_phbs the only way you can control how many PHBs a PowerNV 
>> guest will have?
> 
> yes. Unless we make them user creatable but that's a discussion in
> progress. I am not sure user creatable PHB3s are needed. We will
> see.
>  
>> I would play around and try to figure out all the kinks on my own,
>> but I can't actually compile QEMU with this patch applied:
> 
> you need to use David's branch :
> 
>       https://github.com/dgibson/qemu/tree/ppc-for-3.0

I have pushed a tree here :

        https://github.com/legoater/qemu/tree/phb3-3.0

with two patches, the one we have been discussing about, with some
cleanups and fixes, and one adding user creatable PHB3 devices so 
that extra phbs and devices can be defined on the command line :

-device pnv-phb3,chip-id=1,index=1,id=phb-1.1 -device 
nec-usb-xhci,bus=pci.1,addr=0x7

The PHB identifier is named 'index' now, to be compatible with the 
libvirt attribute. 

Cheers,

C.