Re: [RFC PATCH v2 1/3] vfio: Move the saving of the config space to the right place in VFIO migration

[Kirti Wankhede]

On 1/27/2021 3:06 AM, Alex Williamson wrote:
> On Thu, 10 Dec 2020 10:21:21 +0800
> Shenming Lu <lushenming@huawei.com> wrote:
>
> > On 2020/12/10 2:34, Alex Williamson wrote:
> > > On Wed, 9 Dec 2020 13:29:47 +0100
> > > Cornelia Huck <cohuck@redhat.com> wrote:
> > >    
> > > > On Wed, 9 Dec 2020 16:09:17 +0800
> > > > Shenming Lu <lushenming@huawei.com> wrote:
> > > >   
> > > > > On ARM64 the VFIO SET_IRQS ioctl is dependent on the VM interrupt
> > > > > setup, if the restoring of the VFIO PCI device config space is
> > > > > before the VGIC, an error might occur in the kernel.
> > > > >
> > > > > So we move the saving of the config space to the non-iterable
> > > > > process, so that it will be called after the VGIC according to
> > > > > their priorities.
> > > > >
> > > > > As for the possible dependence of the device specific migration
> > > > > data on it's config space, we can let the vendor driver to
> > > > > include any config info it needs in its own data stream.
> > > > > (Should we note this in the header file linux-headers/linux/vfio.h?)
> > > >
> > > > Given that the header is our primary source about how this interface
> > > > should act, we need to properly document expectations about what will
> > > > be saved/restored when there (well, in the source file in the kernel.)
> > > > That goes in both directions: what a userspace must implement, and what
> > > > a vendor driver can rely on.
> >
> > Yeah, in order to make the vendor driver and QEMU cooperate better, we might
> > need to document some expectations about the data section in the migration
> > region...
> > > > [Related, but not a todo for you: I think we're still missing proper
> > > > documentation of the whole migration feature.]
> > >
> > > Yes, we never saw anything past v1 of the documentation patch.  Thanks,

I'll get back on this and send next version soon.

> > By the way, is there anything unproper with this patch? Wish your suggestion. 
> > :-)
>
> I'm really hoping for some feedback from Kirti, I understand the NVIDIA
> vGPU driver to have some dependency on this.  Thanks,

NVIDIA driver doesn't use device config space value/information during 
device data restoration, so we are good with this change.

Thanks,
Kirti


> Alex
>
> > > > > Signed-off-by: Shenming Lu <lushenming@huawei.com>
> > > > > ---
> > > > >   hw/vfio/migration.c | 25 +++++++++++++++----------
> > > > >   1 file changed, 15 insertions(+), 10 deletions(-)
> > >
> > > .
> > >