Re: [Qemu-devel] [RFC PATCH v4 0/3] Add Mediated device support[was: Add vGPU support]

[Tian, Kevin]

> From: Alex Williamson [mailto:address@hidden
> Sent: Wednesday, May 25, 2016 9:44 PM
> 
> On Wed, 25 May 2016 07:13:58 +0000
> "Tian, Kevin" <address@hidden> wrote:
> 
> > > From: Kirti Wankhede [mailto:address@hidden
> > > Sent: Wednesday, May 25, 2016 3:58 AM
> > >
> > > This series adds Mediated device support to v4.6 Linux host kernel. 
> > > Purpose
> > > of this series is to provide a common interface for mediated device
> > > management that can be used by different devices. This series introduces
> > > Mdev core module that create and manage mediated devices, VFIO based 
> > > driver
> > > for mediated PCI devices that are created by Mdev core module and update
> > > VFIO type1 IOMMU module to support mediated devices.
> >
> > Thanks. "Mediated device" is more generic than previous one. :-)
> >
> > >
> > > What's new in v4?
> > > - Renamed 'vgpu' module to 'mdev' module that represent generic term
> > >   'Mediated device'.
> > > - Moved mdev directory to drivers/vfio directory as this is the extension
> > >   of VFIO APIs for mediated devices.
> > > - Updated mdev driver to be flexible to register multiple types of drivers
> > >   to mdev_bus_type bus.
> > > - Updated mdev core driver with mdev_put_device() and mdev_get_device() 
> > > for
> > >   mediated devices.
> > >
> > >
> >
> > Just curious. In this version you move the whole mdev core under
> > VFIO now. Sorry if I missed any agreement on this change. IIRC Alex
> > doesn't want VFIO to manage mdev life-cycle directly. Instead VFIO is
> > just a mdev driver on created mediated devices....
> 
> I did originally suggest keeping them separate, but as we've progressed
> through the implementation, it's become more clear that the mediated
> device interface is very much tied to the vfio interface, acting mostly
> as a passthrough.  So I thought it made sense to pull them together.
> Still open to discussion of course.  Thanks,
> 

The main benefit of maintaining a separate mdev framework, IMHO, is
to allow better support of both KVM and Xen. Xen doesn't work with VFIO
today, because other VM's memory is not allocated from Dom0 which
means VFIO within Dom0 doesn't has view/permission to control isolation 
for other VMs.

However, after some thinking I think it might not be a big problem to
combine VFIO/mdev together, if we extend Xen to just use VFIO for
resource enumeration. In such model, VFIO still behaves as a single 
kernel portal to enumerate mediated devices to user space, but give up 
permission control to Qemu which will request a secure agent - Xen
hypervisor - to ensure isolation of VM usage on mediated device (including
EPT/IOMMU configuration).

I'm not sure whether VFIO can support this usage today. It is somehow 
similar to channel io passthru in s390, where we also rely on Qemu to 
mediate ccw commands to ensure isolation. Maybe just some slight 
extension is required (e.g. not assume some API must be invoked). Of 
course Qemu side vfio code also need some change. If this can work, 
at least we can first put it as the enumeration interface for mediated 
device in Xen. In the future it may be extended to cover normal Xen 
PCI assignment as well instead of using sysfs to read PCI resource
today.

If above works, then we have a sound plan to enable mediated devices 
based on VFIO first for KVM, and then extend to Xen with reasonable 
effort.
 
How do you think about it?

Thanks
Kevin