Re: [Qemu-devel] [PATCH for-2.9 2/2] intel_iommu: extend supported guest aw to 48 bits

[Alex Williamson]

On Tue, 13 Dec 2016 05:00:07 +0000
"Tian, Kevin" <address@hidden> wrote:

> > From: Alex Williamson
> > Sent: Tuesday, December 13, 2016 3:36 AM
> > 
> > On Mon, 12 Dec 2016 10:01:15 +0800
> > Peter Xu <address@hidden> wrote:
> >   
> > > On Sun, Dec 11, 2016 at 05:13:45AM +0200, Michael S. Tsirkin wrote:  
> > > > On Wed, Dec 07, 2016 at 01:52:45PM +0800, Peter Xu wrote:  
> > > > > Previously vt-d codes only supports 39 bits iova address width. It 
> > > > > won't
> > > > > be hard to extend it to 48 bits.
> > > > >
> > > > > After enabling this, we should be able to map larger iova addresses.
> > > > >
> > > > > To check whether 48 bits aw is enabled, we can grep in the guest dmesg
> > > > > with line: "dmar: Host address width 48" (previously it was 39).
> > > > >
> > > > > Signed-off-by: Peter Xu <address@hidden>  
> > > >
> > > > I suspect we can't do this for old machine types.
> > > > Need to behave in compatible ways.  
> > >
> > > Sure. I can do that.
> > >
> > > Btw, is vt-d iommu still in experimental stage? I am just thinking
> > > whether it'll be overkill we add lots of tunables before we have one
> > > stable and mature vt-d emulation.
> > >  
> > > > Also, is 48 always enough? 5 level with 57 bits
> > > > is just around the corner.  
> > >
> > > Please refer to the discussion with Jason - looks like vt-d spec
> > > currently supports only 39/48 bits address width? Please correct if I
> > > made a mistake.
> > >  
> > > > And is it always supported? for things like vfio
> > > > to work, don't we need to check what does host support?  
> > >
> > > Hmm, yes, we should do that. But until now, we still don't have a
> > > complete vfio support. IMHO we can postpone this issue until vfio is
> > > fully supported.  
> > 
> > I'm not sure how the vIOMMU supporting 39 bits or 48 bits is directly
> > relevant to vfio, we're not sharing page tables.  There is already a
> > case today, without vIOMMU that you can make a guest which has more
> > guest physical address space than the hardware IOMMU by overcommitting
> > system memory.  Generally this quickly resolves itself when we start
> > pinning pages since the physical address width of the IOMMU is
> > typically the same as the physical address width of the host system
> > (ie. we exhaust the host memory).  It is possible though that we could
> > create a sparse memory VM that makes use of gfns beyond the physical
> > IOMMU, with or without a vIOMMU.  You'll get an error from the mapping
> > ioctl when this occurs and the VM will abort.  It's not typically an
> > issue since the memory capacity of the host and the IOMMU physical
> > address width really better align fairly well.  Thanks,
> > 
> > Alex  
> 
> Hi, Alex,
> 
> I have a different thought here regarding to w/ and w/o vIOMMU.
> 
> When there is no vIOMMU exposed, page pinning happens when 
> we assign a device. Exceeding physical IOMMU address weight 
> leads to failure of device assignment. It's just fine.
> 
> However when vIOMMU is exposed, guest IOVA programming 
> happens at run-time. If a IOVA happens to exceed physical address
> width, aborting the VM looks unreasonable since there's nothing
> wrong with guest's operation. We'd better provide a way to at least
> notify guest about such error. Aligning vIOMMU address width to 
> pIOMMU allows us triggering a virtual IOMMU fault - with proper
> error code to indicating such error condition to guest.

QEMU supports memory hotplug, so the first scenario can lead to
aborting the VM runtime as well.  However, I guess the vIOMMU case
really boils down to how the guest makes use of the vIOMMU IOVA space.
The guest could decide to hand out DMA tokens starting at the high end
of the supported vIOMMU width.  Once we enable guest vfio, we allow a
user to manage the IOVA space for a device, so a malicious user within
the guest could abort the entire guest.  So perhaps you're right that
the scenarios available to abort the VM are far more numerous that I
originally thought.

Exposing the host IOMMU address width through the vfio API has been on
my todo list for a long time, it's only been more recently that the
iommu_domain structure includes a geometry structure to expose this
information out through the IOMMU API.  A new capability within the
vfio iommu info data could describe the domain width.  Thanks,

Alex