Re: [Qemu-devel] [PATCH qemu] spapr-pci: Make MMIO spacing a machine property and increase it

[David Gibson]

On Tue, Mar 08, 2016 at 10:50:51AM +1100, Alexey Kardashevskiy wrote:
> On 03/04/2016 03:13 PM, Alexey Kardashevskiy wrote:
> >On 03/04/2016 02:39 PM, David Gibson wrote:
> >>On Thu, Mar 03, 2016 at 12:42:53PM +1100, Alexey Kardashevskiy wrote:
> >>>The pseries machine supports multiple PHBs. Each PHB's MMIO/IO space is
> >>>mapped to the CPU address space starting at SPAPR_PCI_WINDOW_BASE plus
> >>>some offset which is calculated from PHB's index and
> >>>SPAPR_PCI_WINDOW_SPACING which is defined now as 64GB.
> >>>
> >>>Since the default 32bit DMA window is using first 2GB of MMIO space,
> >>>the amount of MMIO which the PCI devices can actually use is reduced
> >>>to 62GB. This is a problem if the user wants to use devices with
> >>>huge BARs.
> >>>
> >>>For example, 2 PCI functions of a NVIDIA K80 adapter being passed through
> >>>will exceed this limit as they have 16M + 16G + 32M BARs which
> >>>(when aligned) will need 64GB.
> >>>
> >>>This converts SPAPR_PCI_WINDOW_BASE and SPAPR_PCI_WINDOW_SPACING to
> >>>sPAPRMachineState properties. This uses old values for pseries machine
> >>>before 2.6 and increases the spacing to 128GB so MMIO space becomes 126GB.
> >>>
> >>>This changes the default value of sPAPRPHBState::mem_win_size to -1 for
> >>>pseries-2.6 and adds setup to spapr_phb_realize.
> >>>
> >>>Signed-off-by: Alexey Kardashevskiy <address@hidden>
> >>
> >>So, in theory I dislike the spapr_pci device reaching into the machine
> >>type to get the spacing configuration.  But.. I don't know of a better
> >>way to achieve the desired outcome.
> >
> >
> >We could drop @index and spacing; and request the user to specify the MMIO
> >window start (at least) for every additional PHB.
> 
> So what is the decision? :)

There isn't one.  I really don't know how to handle this, trying to
talk to some people for ideas.

-- 
David Gibson                    | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
                                | _way_ _around_!
http://www.ozlabs.org/~dgibson

signature.asc
Description: PGP signature