Re: [Qemu-devel] [PATCH 00/17] s390x: the big pci update

[Cornelia Huck]

On Tue, 28 Jun 2016 17:33:58 +0300
Marcel Apfelbaum <address@hidden> wrote:

> On 06/24/2016 04:28 PM, Cornelia Huck wrote:
> > We had been looking at remodelling the pci representation for s390x
> > to handle our slightly odd architecture correctly some time ago
> > already, but now we have a patchset that we're happy with.
> >
> > There's a bunch of bugfixes, cleanups and architecture conformance
> > changes in there, but the most interesting part is the modelling
> > (which, in some respects, takes a cue from what sPAPR does).
> >
> > We introduce a 'zpci' device to hold s390x-specific properties like
> > the uid and fid. This 'zpci' device is connected to a run-of-the-mill
> > pci device via the 'target' property.
> >
> > Example command line portion:
> >
> > -device zpci,uid=1,fid=1,target=vpci0 \
> > -device vfio-pci,host=0000:00:00.0,id=vpci0 \
> > -device zpci,target=vpci1 \
> > -device vfio-pci,host=0001:00:00.0,id=vpci1 \
> > -device vfio-pci,host=0002:00:00.0,id=vpci2
> >
> > For device vpci0, uid and fid are specified in the associated zpci
> > device.
> > For device vpci1, uid and fid are automatically generated.
> > For device vpci2, first an associated zpci device is generated and
> > then autogenerated values for uid and fid are placed in it.
> >
> > This should accomodate both specifying our special parameters and
> > re-using standard statements for pci devices.
> 
> [...]
> 
>   You can still specify
> > bus/slot/function for the pci device, but it will not be propagated
> > into the guest (as the s390 pci architecture does not allow for it;
> > the guest only sees uid/fid).
> >
> 
> Hi,
> 
> Please excuse my lack of s390 knowledge, but I find hard to understand
> how it works. If using bus/slot/function is not an option,
> how can one access the configuration space? This configuration is not PCI 
> compliant?

Yes, pci on s390 is... odd. Basically, we have some special
instructions for it.

For device discovery, the guest OS uses an instruction CLP (with a
subfunction) to list the pci functions available: see list_pci() in
hw/s390x/s390-pci-inst.c. This list contains the function handle (fh)
and the fid for each function.

For nearly all subsequent operations that target a pci function (like
config space accesses), the guest OS will use other instructions that
take the fh as an argument (see also hw/s390x/s390-pci-inst.c, for
example pci{lg,stg}_service_call() for reading/writing). The fid is
used for enabling/disabling via yet another instruction.

As an example of how this is used (and how one may construct a pci
"topology" from this), see arch/s390/pci/ in the Linux kernel. That's
why we have bus/slot/function in the host, even though they are
entirely synthetic.