Re: [Qemu-devel] [PATCH 0/5] QEMU VFIO live migration

[Alex Williamson]

On Mon, 1 Apr 2019 04:40:03 -0400
Yan Zhao <address@hidden> wrote:

> On Mon, Apr 01, 2019 at 04:14:30PM +0800, Cornelia Huck wrote:
> > On Wed, 27 Mar 2019 16:10:20 -0600
> > Alex Williamson <address@hidden> wrote:
> >   
> > > On Wed, 27 Mar 2019 20:18:54 +0000
> > > "Dr. David Alan Gilbert" <address@hidden> wrote:
> > >   
> > > > * Zhao Yan (address@hidden) wrote:    
> > > > > On Wed, Feb 20, 2019 at 07:42:42PM +0800, Cornelia Huck wrote:      
> > > > > > > > > >   b) How do we detect if we're migrating from/to the wrong 
> > > > > > > > > > device or
> > > > > > > > > > version of device?  Or say to a device with older firmware 
> > > > > > > > > > or perhaps
> > > > > > > > > > a device that has less device memory ?        
> > > > > > > > > Actually it's still an open for VFIO migration. Need to think 
> > > > > > > > > about
> > > > > > > > > whether it's better to check that in libvirt or qemu (like a 
> > > > > > > > > device magic
> > > > > > > > > along with verion ?).        
> > > > > > > 
> > > > > > > We must keep the hardware generation is the same with one POD of 
> > > > > > > public cloud
> > > > > > > providers. But we still think about the live migration between 
> > > > > > > from the the lower
> > > > > > > generation of hardware migrated to the higher generation.      
> > > > > > 
> > > > > > Agreed, lower->higher is the one direction that might make sense to
> > > > > > support.
> > > > > > 
> > > > > > But regardless of that, I think we need to make sure that 
> > > > > > incompatible
> > > > > > devices/versions fail directly instead of failing in a subtle, hard 
> > > > > > to
> > > > > > debug way. Might be useful to do some initial sanity checks in 
> > > > > > libvirt
> > > > > > as well.
> > > > > > 
> > > > > > How easy is it to obtain that information in a form that can be
> > > > > > consumed by higher layers? Can we find out the device type at least?
> > > > > > What about some kind of revision?      
> > > > > hi Alex and Cornelia
> > > > > for device compatibility, do you think it's a good idea to use 
> > > > > "version"
> > > > > and "device version" fields?
> > > > > 
> > > > > version field: identify live migration interface's version. it can 
> > > > > have a
> > > > > sort of backward compatibility, like target machine's version >= 
> > > > > source
> > > > > machine's version. something like that.    
> > > 
> > > Don't we essentially already have this via the device specific region?
> > > The struct vfio_info_cap_header includes id and version fields, so we
> > > can declare a migration id and increment the version for any
> > > incompatible changes to the protocol.
> > >   
> > > > > 
> > > > > device_version field consists two parts:
> > > > > 1. vendor id : it takes 32 bits. e.g. 0x8086.    
> > > 
> > > Who allocates IDs?  If we're going to use PCI vendor IDs, then I'd
> > > suggest we use a bit to flag it as such so we can reserve that portion
> > > of the 32bit address space.  See for example:
> > > 
> > > #define VFIO_REGION_TYPE_PCI_VENDOR_TYPE        (1 << 31)
> > > #define VFIO_REGION_TYPE_PCI_VENDOR_MASK        (0xffff)
> > > 
> > > For vendor specific regions.  
> > 
> > Just browsing through the thread... if I don't misunderstand, we could
> > use a vfio-ccw region type id here for ccw, couldn't we? Just to make
> > sure that this is not pci-specific.  
> CCW could use another bit other than bit 31?
> e.g.
> #define VFIO_REGION_TYPE_CCW_VENDOR_TYPE        (1 << 30)
> then ccw device use (VFIO_REGION_TYPE_CCW_VENDOR_TYPE | vendor id) as its
> first 32 bit for device version string.
> 
> But as Alex said we'll not provide an extra region to get device version,
> and device version is only exported in sysfs, probably we should define them 
> as
> below:
> #define VFIO_DEVICE_VERSION_TYPE_PCI (1<<31)
> #define VFIO_DEVICE_VERSION_TYPE_CCW (1<<30)
> 
> Do you think it's ok?

We already had this discussion for device specific regions and decided
that CCW doesn't have enough vendors to justify a full subset of the
available address space.  Also, this doesn't need to imply the device
interface, we're simply specifying a vendor registrar such that we can
give each vendor their own namespace, so I don't think it would be a
problem for a CCW to specify a namespace using a PCI vendor ID.
Finally, if we have such need for this in the future, because I'm not
sure where we stand with this in the current proposals, maybe we should
make use of an IEEE OUI rather than a PCI database to avoid this sort
of confusion and mis-association if we have further need.  Thanks,

Alex