[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH for-2.10 0/3] qdev/vfio: defer DEVICE_DEL to avo
From: |
Auger Eric |
Subject: |
Re: [Qemu-devel] [PATCH for-2.10 0/3] qdev/vfio: defer DEVICE_DEL to avoid races with libvirt |
Date: |
Wed, 9 Aug 2017 16:53:07 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 |
Hi Michael,
On 27/07/2017 03:30, Michael Roth wrote:
> This series was motivated by the discussion in this thread:
>
> https://www.redhat.com/archives/libvir-list/2017-June/msg01370.html
>
> The issue this series addresses is that when libvirt unplugs a VFIO PCI
> device,
> it may attempt to bind the host device back to the host driver when QEMU emits
> the DEVICE_DELETED event for the corresponding vfio-pci device. However, the
> VFIO group FD is not actually cleaned up until vfio-pci device is *finalized*
> by QEMU, whereas the event is emitted earlier during device_unparent.
> Depending on the host device and how long certain operations like resetting
> the
> device might take, this can in result in libvirt trying to rebind the device
> back to the host while it is still in use by VFIO, leading to host crashes or
> other unexpected behavior.
>
> In particular, Mellanox CX4 adapters on PowerNV hosts might not be fully
> quiesced by vfio-pci's finalize() routine until up to 6s after the
> DEVICE_DELETED was emitted, leading to detach-device on the libvirt side
> pretty
> much always crashing the host.
>
> Implementing this change requires 2 prereqs to ensure the same information is
> available when the DEVICE_DELETED is finally emitted:
>
> 1) Storing the path in the composition patch, which is addressed by PATCH 1,
> which was plucked from another pending series from Greg Kurz:
>
> https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg07922.html
>
> since we are now "disconnected" at the time the event is emitted, and
>
> 2) Deferring qemu_opts_del of the DeviceState->QemuOpts till finalize, since
> that is where DeviceState->id is stored. This was actually how it was
> done in the past, so PATCH 2 simply reverts the change which moved it to
> device_unparent.
>
> From there it's just a mechanical move of the event from device_unparent to
> device_finalize.
>
> hw/core/qdev.c | 30 +++++++++++++++++++-----------
> include/hw/qdev-core.h | 1 +
> 2 files changed, 20 insertions(+), 11 deletions(-)
>
>
Tested-by: Eric Auger <address@hidden>
Without the series I get the following warning when detaching a pci host
dev from virt-manager:
2017-08-09T14:28:44.825925Z qemu-system-aarch64: Failed to remove group
53 from KVM VFIO device: No such file or directory
as KVM_DEV_VFIO_GROUP_DEL returns -ENOENT due to the fact the fd for the
VFIO group does not exist anymore as libvirt already started to unbind
the device from vfio-pci at vfio_instance_finalize() time.
With the series I don't see the message anymore.
Thanks
Eric