Re: [PATCH v4 28/41] vfio/iommufd: Implement the iommufd backend

[Matthew Rosato]

On 11/2/23 3:12 AM, Zhenzhong Duan wrote:
> From: Yi Liu <yi.l.liu@intel.com>
> 
> Add the iommufd backend. The IOMMUFD container class is implemented
> based on the new /dev/iommu user API. This backend obviously depends
> on CONFIG_IOMMUFD.
> 
> So far, the iommufd backend doesn't support dirty page sync yet due
> to missing support in the host kernel.
> 
> Co-authored-by: Eric Auger <eric.auger@redhat.com>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> Signed-off-by: Yi Liu <yi.l.liu@intel.com>
> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
> ---

[...]

> +static int iommufd_cdev_attach_container(VFIODevice *vbasedev,
> +                                         VFIOIOMMUFDContainer *container,
> +                                         Error **errp)
> +{
> +    int ret, iommufd = vbasedev->iommufd->fd;
> +    VFIOIOASHwpt *hwpt;
> +    uint32_t hwpt_id;
> +    Error *err = NULL;
> +
> +    /* try to attach to an existing hwpt in this container */
> +    QLIST_FOREACH(hwpt, &container->hwpt_list, next) {
> +        ret = iommufd_cdev_attach_hwpt(vbasedev, hwpt->hwpt_id, &err);
> +        if (ret) {
> +            const char *msg = error_get_pretty(err);
> +
> +            trace_iommufd_cdev_fail_attach_existing_hwpt(msg);
> +            error_free(err);
> +            err = NULL;
> +        } else {
> +            goto found_hwpt;
> +        }
> +    }
> +
> +    ret = iommufd_backend_alloc_hwpt(iommufd, vbasedev->devid,
> +                                     container->ioas_id, &hwpt_id);
> +
> +    if (ret) {
> +        error_setg_errno(errp, errno, "error alloc shadow hwpt");
> +        return ret;
> +    }

The above alloc_hwpt fails for mdevs (at least, it fails for me attempting to 
use iommufd backend with vfio-ccw and vfio-ap on s390).  The ioctl is failing 
in the kernel because it can't find an IOMMUFD_OBJ_DEVICE.

AFAIU that's because the mdevs are meant to instead use kernel access via 
vfio_iommufd_emulated_attach_ioas, not hwpt.  That's how mdevs behave when 
looking at the kernel vfio compat container.

As a test, I was able to get vfio-ccw and vfio-ap working using the iommufd 
backend by just skipping this alloc_hwpt above and instead passing 
container->ioas_id into the iommufd_cdev_attach_hwpt below.  That triggers the 
vfio_iommufd_emulated_attach_ioas call in the kernel.

> +
> +    /* Attach cdev to a new allocated hwpt within iommufd */
> +    ret = iommufd_cdev_attach_hwpt(vbasedev, hwpt_id, errp);
> +    if (ret) {
> +        iommufd_backend_free_id(iommufd, hwpt_id);
> +        return ret;
> +    }
> +
> +    hwpt = iommufd_container_get_hwpt(container, hwpt_id);
> +found_hwpt:
> +    QLIST_INSERT_HEAD(&hwpt->device_list, vbasedev, next);
> +    vbasedev->hwpt = hwpt;
> +
> +    trace_iommufd_cdev_attach_container(iommufd, vbasedev->name, 
> vbasedev->fd,
> +                                        container->ioas_id, hwpt->hwpt_id);
> +    return ret;
> +}