Re: [Qemu-devel] [PULL 08/41] intel_iommu: support device iotlb descriptor

[Liu, Yi L]

> -----Original Message-----
> From: Jason Wang [mailto:address@hidden
> Sent: Monday, February 20, 2017 5:04 PM
> To: Liu, Yi L <address@hidden>; Michael S. Tsirkin <address@hidden>; qemu-
> address@hidden
> Cc: Lan, Tianyu <address@hidden>; Peter Maydell
> <address@hidden>; Tian, Kevin <address@hidden>; Eduardo
> Habkost <address@hidden>; Peter Xu <address@hidden>; Alex
> Williamson <address@hidden>; Paolo Bonzini
> <address@hidden>; Richard Henderson <address@hidden>
> Subject: Re: [Qemu-devel] [PULL 08/41] intel_iommu: support device iotlb
> descriptor
> 
> 
> 
> On 2017年02月20日 16:27, Liu, Yi L wrote:
> >> -----Original Message-----
> >> From: Jason Wang [mailto:address@hidden
> >> Sent: Friday, February 17, 2017 2:43 PM
> >> To: Liu, Yi L <address@hidden>; Michael S. Tsirkin
> >> <address@hidden>; qemu- address@hidden
> >> Cc: Peter Maydell <address@hidden>; Eduardo Habkost
> >> <address@hidden>; Peter Xu <address@hidden>; Paolo Bonzini
> >> <address@hidden>; Richard Henderson <address@hidden>; Tian,
> >> Kevin <address@hidden>; Lan, Tianyu <address@hidden>;
> >> Alex Williamson <address@hidden>
> >> Subject: Re: [Qemu-devel] [PULL 08/41] intel_iommu: support device
> >> iotlb descriptor
> >>
> >>
> >>
> >> On 2017年02月17日 14:18, Liu, Yi L wrote:
> >>>> -----Original Message-----
> >>>> From: Jason Wang [mailto:address@hidden
> >>>> Sent: Thursday, February 16, 2017 1:44 PM
> >>>> To: Liu, Yi L <address@hidden>; Michael S. Tsirkin
> >>>> <address@hidden>; qemu- address@hidden
> >>>> Cc: Peter Maydell <address@hidden>; Eduardo Habkost
> >>>> <address@hidden>; Peter Xu <address@hidden>; Paolo Bonzini
> >>>> <address@hidden>; Richard Henderson <address@hidden>; Tian,
> >>>> Kevin <address@hidden>; Lan, Tianyu <address@hidden>;
> >>>> Alex Williamson <address@hidden>
> >>>> Subject: Re: [Qemu-devel] [PULL 08/41] intel_iommu: support device
> >>>> iotlb descriptor
> >>>>
> >>>>
> >>>>
> >>>> On 2017年02月16日 13:36, Liu, Yi L wrote:
> >>>>>> -----Original Message-----
> >>>>>> From: Qemu-devel
> >>>>>> [mailto:address@hidden
> >>>>>> On Behalf Of Michael S. Tsirkin
> >>>>>> Sent: Tuesday, January 10, 2017 1:40 PM
> >>>>>> To: address@hidden
> >>>>>> Cc: Peter Maydell <address@hidden>; Eduardo Habkost
> >>>>>> <address@hidden>; Jason Wang <address@hidden>; Peter
> >> Xu
> >>>>>> <address@hidden>; Paolo Bonzini <address@hidden>; Richard
> >>>>>> Henderson <address@hidden>
> >>>>>> Subject: [Qemu-devel] [PULL 08/41] intel_iommu: support device
> >>>>>> iotlb descriptor
> >>>>>>
> >>>>>> From: Jason Wang <address@hidden>
> >>>>>>
> >>>>>> This patch enables device IOTLB support for intel iommu. The
> >>>>>> major work is to implement QI device IOTLB descriptor processing
> >>>>>> and notify the device through iommu notifier.
> >>>>>>
> >>>>> Hi Jason/Michael,
> >>>>>
> >>>>> Recently Peter Xu's patch also touched intel-iommu emulation. His
> >>>>> patch shadows second-level page table by capturing iotlb flush
> >>>>> from guest. It would result in page table updating in host. Does
> >>>>> this patch also use the same map/umap API provided by VFIO?
> >>>> Yes, it depends on the iommu notifier too.
> >>>>
> >>>>> If it is, then I think it would also update page table in host. It
> >>>>> looks to be a duplicate update. Pls refer to the following
> >>>>> snapshot captured from section 6.5.2.5 of vtd spec.
> >>>>>
> >>>>> "Since translation requests from a device may be serviced by
> >>>>> hardware from the IOTLB, software must always request IOTLB
> >>>>> invalidation
> >>>>> (iotlb_inv_dsc) before requesting corresponding Device-TLB
> >>>>> (dev_tlb_inv_dsc) invalidation."
> >>>>>
> >>>>> Maybe for device-iotlb, we need a separate API which just pass
> >>>>> down the invalidate info without updating page table. Any thoughts?
> >>>> cc Alex.
> >>>>
> >>>> If we want ATS to be visible for guest (but I'm not sure if VFIO
> >>>> support this), we probably need another notifier or a new flag.
> >>> Jason, for assigned device, I think guest could see ATS if the
> >>> assigned device supports ATS. I can see it when passthru iGPU.
> >>>
> >>> Regards,
> >>> Yi L
> >> Good to know this.
> >>
> >> If I understand your suggestion correctly, you want a dedicated API
> >> to flush a hardware device IOTLB. I'm not sure this is really needed.
> > yes, I'd like to have an extra API besides the current MAP/UNMAP.
> 
> I'm think whether or not we can do this without extra API or even don't need 
> to
> care about this.
> 
> >
> >> There's some discussion of similar issue in the past (when ATS is
> >> used for virtio- net/vhost), looks like we could solve this by not
> >> trigger the UNMAP notifier unless it was device IOTLB inv desc if ATS
> >> is enabled for the device? With this remote IOMMU/IOTLB can only get
> >> invalidation request once. For VFIO, the under layer IOMMU can deal
> >> with hardware device IOTLB without any extra efforts.
> > If I catch the background, I think it should be "not trigger the UNMAP
> > notifier when unless it was device IOTLB inv desc if ATS is enabled for the
> device"
> 
> Seems not :)
> 
> I mean, if ATS is enabled for the device, only trigger UNMAP notifier when
> processing device IOTLB. Then we can only have flush once. And host IOMMU
> driver will take care of device IOTLB flush too.
hmmm, how about the iotlb inv desc which is prior to device-iotlb? I'm not sure
if it is practical to ignore the iotlb inv des since there is no SID info in it.

Regards,
Yi L