Re: [RFC PATCH for 8.0 00/13] vDPA-net inflight descriptors migration with SVQ

[Jason Wang]

On Tue, Dec 6, 2022 at 1:04 AM Eugenio Pérez <eperezma@redhat.com> wrote:
>
> The state of the descriptors (avail or used) may not be recoverable just
> looking at the guest memory.  Out of order used descriptor may override
> previous avail ones in the descriptor table or avail vring.
>
> Currently we're not migrating this status in net devices because virtio-net,
> vhost-kernel etc use the descriptors in order,

Note that this might not be the truth (when zerocopy is enabled).

> so the information always
> recoverable from guest's memory.  However, vDPA devices may use them out of
> order, and other kind of devices like block need this support.
>
> Shadow virtqueue is able to track these and resend them at the destination.

As discussed, there's a bootstrap issue here:

When SVQ needs to be enabled on demand, do we still need another way
to get inflight ones without the help of SVQ?

Thanks

> Add them to the virtio-net migration description so they are not lose in the
> process.
>
> This is a very early RFC just to validate the first draft so expect leftovers.
> To fetch and request the descriptors from a device without SVQ need to be
> implemented on top. Some other notable pending items are:
> * Do not send the descriptors actually recoverable from the guest memory.
> * Properly version the migrate data.
> * Properly abstract the descriptors access from virtio-net to SVQ.
> * Do not use VirtQueueElementOld but migrate directly VirtQueueElement.
> * Replace lots of assertions with runtime conditionals.
> * Other TODOs in the patch message or code changes.
>
> Thanks.
>
> Eugenio Pérez (13):
>   vhost: add available descriptor list in SVQ
>   vhost: iterate only available descriptors at SVQ stop
>   vhost: merge avail list and next avail descriptors detach
>   vhost: add vhost_svq_save_inflight
>   virtio: Specify uint32_t as VirtQueueElementOld members type
>   virtio: refactor qemu_get_virtqueue_element
>   virtio: refactor qemu_put_virtqueue_element
>   virtio: expose VirtQueueElementOld
>   virtio: add vmstate_virtqueue_element_old
>   virtio-net: Migrate vhost inflight descriptors
>   virtio-net: save inflight descriptors at vhost shutdown
>   vhost: expose vhost_svq_add_element
>   vdpa: Recover inflight descriptors
>
>  hw/virtio/vhost-shadow-virtqueue.h |   9 ++
>  include/hw/virtio/virtio-net.h     |   2 +
>  include/hw/virtio/virtio.h         |  32 ++++++
>  include/migration/vmstate.h        |  22 ++++
>  hw/net/vhost_net.c                 |  56 ++++++++++
>  hw/net/virtio-net.c                | 129 +++++++++++++++++++++++
>  hw/virtio/vhost-shadow-virtqueue.c |  52 +++++++--
>  hw/virtio/vhost-vdpa.c             |  11 --
>  hw/virtio/virtio.c                 | 162 ++++++++++++++++++-----------
>  9 files changed, 392 insertions(+), 83 deletions(-)
>
> --
> 2.31.1
>
>