Re: [RFC PATCH] Allow VIRTIO_F_IN_ORDER to be negotiated for vdpa devices

[Jason Wang]

On Sun, Feb 6, 2022 at 12:04 AM <gautam.dawar@xilinx.com> wrote:
>
> From: Gautam Dawar <gdawar@xilinx.com>
>
> Hi All,
>
> The VIRTIO_F_IN_ORDER feature is implemented by DPDK's virtio_net
> driver but not by the Linux kernel's virtio_net driver.
> However, this feature still can't be tested using vhost-vdpa with
> hardware devices that implement it as VIRTIO_F_IN_ORDER isn't defined
> in kernel's virtio_config.h header file.
> This patch adds the ability to negotiate VIRTIO_F_IN_ORDER bit for
> vhost-vdpa backend when the underlying device supports this feature.
> This would be useful for benchmarking the performance improvements
> for HW devices that implement this feature. At the same time, it
> shouldn't have any negative impact as vhost-vdpa backend doesn't
> involve any userspace virtqueue operations.
> In the final patch, instead of making a direct change in
> virtio_config.h, it will be pushed in the kernel and then QEMU's
> file will be synced with it, as usual.
>
> Signed-off-by: Gautam Dawar <gdawar@xilinx.com>
> ---
>  hw/net/virtio-net.c                            | 10 ++++++++++
>  include/standard-headers/linux/virtio_config.h |  6 ++++++
>  net/vhost-vdpa.c                               |  1 +
>  3 files changed, 17 insertions(+)
>
> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
> index cf8ab0f8af..a1089d06f6 100644
> --- a/hw/net/virtio-net.c
> +++ b/hw/net/virtio-net.c
> @@ -3507,11 +3507,21 @@ static void virtio_net_device_realize(DeviceState 
> *dev, Error **errp)
>      nc->rxfilter_notify_enabled = 1;
>
>     if (nc->peer && nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_VDPA) {
> +        uint64_t features = BIT_ULL(VIRTIO_F_IN_ORDER);
>          struct virtio_net_config netcfg = {};
> +
>          memcpy(&netcfg.mac, &n->nic_conf.macaddr, ETH_ALEN);
>          vhost_net_set_config(get_vhost_net(nc->peer),
>              (uint8_t *)&netcfg, 0, ETH_ALEN, VHOST_SET_CONFIG_TYPE_MASTER);
> +
> +       /*
> +         * For vhost-vdpa, if underlying device supports IN_ORDER feature,
> +         * make it available for negotiation.
> +         */
> +       features = vhost_net_get_features(get_vhost_net(nc->peer), features);
> +       n->host_features |= features;

I wonder, instead of doing hacks here, it would be better to implement
IN_ORDER in qemu?

Note that DPDK has already supported IN_ORDER, so we don't even need
to touch kernel virtio drivers to test it.

>      }
> +
>      QTAILQ_INIT(&n->rsc_chains);
>      n->qdev = dev;
>
> diff --git a/include/standard-headers/linux/virtio_config.h 
> b/include/standard-headers/linux/virtio_config.h
> index 22e3a85f67..9ec3a8b54b 100644
> --- a/include/standard-headers/linux/virtio_config.h
> +++ b/include/standard-headers/linux/virtio_config.h
> @@ -80,6 +80,12 @@
>  /* This feature indicates support for the packed virtqueue layout. */
>  #define VIRTIO_F_RING_PACKED           34
>
> +/*
> + * Inorder feature indicates that all buffers are used by the device
> + * in the same order in which they have been made available.
> + */
> +#define VIRTIO_F_IN_ORDER 35

This need to be done in the following steps:

1) a kernel patch to just add this feature bit
2) sync the kernel header using scripts/update-linux-headers.sh

> +
>  /*
>   * This feature indicates that memory accesses by the driver and the
>   * device are ordered in a way described by the platform.
> diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c
> index 25dd6dd975..2886cba5ec 100644
> --- a/net/vhost-vdpa.c
> +++ b/net/vhost-vdpa.c
> @@ -62,6 +62,7 @@ const int vdpa_feature_bits[] = {
>      VIRTIO_NET_F_CTRL_VQ,
>      VIRTIO_F_IOMMU_PLATFORM,
>      VIRTIO_F_RING_PACKED,
> +    VIRTIO_F_IN_ORDER,
>      VIRTIO_NET_F_RSS,
>      VIRTIO_NET_F_HASH_REPORT,
>      VIRTIO_NET_F_GUEST_ANNOUNCE,

This needs to be done in a separated patch.

Thanks

> --
> 2.30.1
>