Re: [Qemu-devel] [PULL 08/41] intel_iommu: support device iotlb descriptor

[Jason Wang]

On 2017年02月20日 16:27, Liu, Yi L wrote:
> > -----Original Message-----
> > From: Jason Wang [mailto:address@hidden
> > Sent: Friday, February 17, 2017 2:43 PM
> > To: Liu, Yi L <address@hidden>; Michael S. Tsirkin <address@hidden>; qemu-
> > address@hidden
> > Cc: Peter Maydell <address@hidden>; Eduardo Habkost
> > <address@hidden>; Peter Xu <address@hidden>; Paolo Bonzini
> > <address@hidden>; Richard Henderson <address@hidden>; Tian, Kevin
> > <address@hidden>; Lan, Tianyu <address@hidden>; Alex Williamson
> > <address@hidden>
> > Subject: Re: [Qemu-devel] [PULL 08/41] intel_iommu: support device iotlb
> > descriptor
> >
> >
> >
> > On 2017年02月17日 14:18, Liu, Yi L wrote:
> > > > -----Original Message-----
> > > > From: Jason Wang [mailto:address@hidden
> > > > Sent: Thursday, February 16, 2017 1:44 PM
> > > > To: Liu, Yi L <address@hidden>; Michael S. Tsirkin
> > > > <address@hidden>; qemu- address@hidden
> > > > Cc: Peter Maydell <address@hidden>; Eduardo Habkost
> > > > <address@hidden>; Peter Xu <address@hidden>; Paolo Bonzini
> > > > <address@hidden>; Richard Henderson <address@hidden>; Tian,
> > > > Kevin <address@hidden>; Lan, Tianyu <address@hidden>;
> > > > Alex Williamson <address@hidden>
> > > > Subject: Re: [Qemu-devel] [PULL 08/41] intel_iommu: support device
> > > > iotlb descriptor
> > > >
> > > >
> > > >
> > > > On 2017年02月16日 13:36, Liu, Yi L wrote:
> > > > > > -----Original Message-----
> > > > > > From: Qemu-devel
> > > > > > [mailto:address@hidden
> > > > > > On Behalf Of Michael S. Tsirkin
> > > > > > Sent: Tuesday, January 10, 2017 1:40 PM
> > > > > > To: address@hidden
> > > > > > Cc: Peter Maydell <address@hidden>; Eduardo Habkost
> > > > > > <address@hidden>; Jason Wang <address@hidden>; Peter
> > Xu
> > > > > > <address@hidden>; Paolo Bonzini <address@hidden>; Richard
> > > > > > Henderson <address@hidden>
> > > > > > Subject: [Qemu-devel] [PULL 08/41] intel_iommu: support device
> > > > > > iotlb descriptor
> > > > > >
> > > > > > From: Jason Wang <address@hidden>
> > > > > >
> > > > > > This patch enables device IOTLB support for intel iommu. The major
> > > > > > work is to implement QI device IOTLB descriptor processing and
> > > > > > notify the device through iommu notifier.
> > > > > Hi Jason/Michael,
> > > > >
> > > > > Recently Peter Xu's patch also touched intel-iommu emulation. His
> > > > > patch shadows second-level page table by capturing iotlb flush from
> > > > > guest. It would result in page table updating in host. Does this
> > > > > patch also use the same map/umap API provided by VFIO?
> > > > Yes, it depends on the iommu notifier too.
> > > >
> > > > > If it is, then I think it would also update page table in host. It
> > > > > looks to be a duplicate update. Pls refer to the following snapshot
> > > > > captured from section 6.5.2.5 of vtd spec.
> > > > >
> > > > > "Since translation requests from a device may be serviced by
> > > > > hardware from the IOTLB, software must always request IOTLB
> > > > > invalidation
> > > > > (iotlb_inv_dsc) before requesting corresponding Device-TLB
> > > > > (dev_tlb_inv_dsc) invalidation."
> > > > >
> > > > > Maybe for device-iotlb, we need a separate API which just pass down
> > > > > the invalidate info without updating page table. Any thoughts?
> > > > cc Alex.
> > > >
> > > > If we want ATS to be visible for guest (but I'm not sure if VFIO
> > > > support this), we probably need another notifier or a new flag.
> > > Jason, for assigned device, I think guest could see ATS if the
> > > assigned device supports ATS. I can see it when passthru iGPU.
> > >
> > > Regards,
> > > Yi L
> > Good to know this.
> >
> > If I understand your suggestion correctly, you want a dedicated API to flush a
> > hardware device IOTLB. I'm not sure this is really needed.
> yes, I'd like to have an extra API besides the current MAP/UNMAP.

I'm think whether or not we can do this without extra API or even don't 
need to care about this.

> > There's some discussion of similar issue in the past (when ATS is used for 
> > virtio-
> > net/vhost), looks like we could solve this by not trigger the UNMAP notifier
> > unless it was device IOTLB inv desc if ATS is enabled for the device? With this
> > remote IOMMU/IOTLB can only get invalidation request once. For VFIO, the
> > under layer IOMMU can deal with hardware device IOTLB without any extra
> > efforts.
> If I catch the background, I think it should be "not trigger the UNMAP notifier
> when unless it was device IOTLB inv desc if ATS is enabled for the device"

Seems not :)

I mean, if ATS is enabled for the device, only trigger UNMAP notifier 
when processing device IOTLB. Then we can only have flush once. And host 
IOMMU driver will take care of device IOTLB flush too.

Thanks

> Feel free to correct me.
>
> Thanks,
> Yi L
> > Does this make sense?
> >
> > Thanks