Enable vfio-pci devices to be saved and restored across a cpr-exec of qemu.
At vfio creation time, save the value of vfio container, group, and device
descriptors in CPR state.
In the container pre_save handler, suspend the use of virtual addresses
in DMA mappings with VFIO_DMA_UNMAP_FLAG_VADDR, because guest ram will
be remapped at a different VA after exec. DMA to already-mapped pages
continues. Save the msi message area as part of vfio-pci vmstate, and
save the interrupt and notifier eventfd's in vmstate.
On qemu restart, vfio_realize() finds the saved descriptors, uses the
descriptors, and notes that the device is being reused. Device and iommu
state is already configured, so operations in vfio_realize that would
modify the configuration are skipped for a reused device, including vfio
ioctl's and writes to PCI configuration space. Vfio PCI device reset
is also suppressed. The result is that vfio_realize constructs qemu
data structures that reflect the current state of the device. However,
the reconstruction is not complete until migrate_incoming is called.
migrate_incoming loads the msi data, the vfio post_load handler finds
eventfds in CPR state, rebuilds vector data structures, and attaches the
interrupts to the new KVM instance. The container post_load handler then
invokes the main vfio listener callback, which walks the flattened ranges
of the vfio address space and calls VFIO_DMA_MAP_FLAG_VADDR to inform the
kernel of the new VA's. Lastly, migration resumes the VM.
This functionality is delivered by 3 patches for clarity. Part 1 handles
device file descriptors and DMA. Part 2 adds eventfd and MSI/MSI-X vector
support. Part 3 adds INTX support.
[...]
diff --git a/hw/vfio/cpr-legacy.c b/hw/vfio/cpr-legacy.c
new file mode 100644
index 0000000..bc51ebe
--- /dev/null
+++ b/hw/vfio/cpr-legacy.c
[...]
+
+bool vfio_legacy_cpr_register_container(VFIOContainerBase *bcontainer,
+ Error **errp)
+{
+ VFIOContainer *container = VFIO_CONTAINER(bcontainer);
+
+ if (!vfio_can_cpr_exec(container, &bcontainer->cpr_blocker)) {
+ return migrate_add_blocker_modes(&bcontainer->cpr_blocker, errp,
+ MIG_MODE_CPR_EXEC, -1);