RE: [PATCH v6 01/21] backends/iommufd: Introduce the iommufd object

[Duan, Zhenzhong]

>-----Original Message-----
>From: Cédric Le Goater <clg@redhat.com>
>Sent: Friday, November 17, 2023 9:56 PM
>Subject: Re: [PATCH v6 01/21] backends/iommufd: Introduce the iommufd object
>
>On 11/17/23 14:29, Eric Auger wrote:
>> Hi Cédric,
>>
>> On 11/17/23 12:39, Duan, Zhenzhong wrote:
>>> Hi Cédric,
>>>
>>>> -----Original Message-----
>>>> From: Cédric Le Goater <clg@redhat.com>
>>>> Sent: Friday, November 17, 2023 7:10 PM
>>>> Subject: Re: [PATCH v6 01/21] backends/iommufd: Introduce the iommufd
>object
>>>>
>>>> Hello,
>>>>
>>>>> +int iommufd_backend_map_dma(IOMMUFDBackend *be, uint32_t ioas_id,
>>>> hwaddr iova,
>>>>> +                            ram_addr_t size, void *vaddr, bool readonly)
>>>>> +{
>>>>> +    int ret, fd = be->fd;
>>>>> +    struct iommu_ioas_map map = {
>>>>> +        .size = sizeof(map),
>>>>> +        .flags = IOMMU_IOAS_MAP_READABLE |
>>>>> +                 IOMMU_IOAS_MAP_FIXED_IOVA,
>>>>> +        .ioas_id = ioas_id,
>>>>> +        .__reserved = 0,
>>>>> +        .user_va = (uintptr_t)vaddr,
>>>>> +        .iova = iova,
>>>>> +        .length = size,
>>>>> +    };
>>>>> +
>>>>> +    if (!readonly) {
>>>>> +        map.flags |= IOMMU_IOAS_MAP_WRITEABLE;
>>>>> +    }
>>>>> +
>>>>> +    ret = ioctl(fd, IOMMU_IOAS_MAP, &map);
>>>>> +    trace_iommufd_backend_map_dma(fd, ioas_id, iova, size,
>>>>> +                                  vaddr, readonly, ret);
>>>>> +    if (ret) {
>>>>> +        ret = -errno;
>>>>> +        error_report("IOMMU_IOAS_MAP failed: %m");
>>>>> +    }
>>>>> +    return ret;
>>>>> +}
>>>> When using a UEFI guest, QEMU reports errors when mapping regions
>>>> in the top PCI space :
>>>>
>>>>    iommufd_backend_map_dma  iommufd=10 ioas=2 iova=0x380000001000
>>>> size=0x3000 addr=0x7fce2c28b000 readonly=0 (-1)
>>>>    qemu-system-x86_64: IOMMU_IOAS_MAP failed: Invalid argument
>>>>    qemu-system-x86_64: vfio_container_dma_map(0x55a21b03a150,
>>>> 0x380000001000, 0x3000, 0x7fce2c28b000) = -22 (Invalid argument)
>>>>
>>>>    iommufd_backend_map_dma  iommufd=10 ioas=2 iova=0x380000004000
>>>> size=0x4000 addr=0x7fce2c980000 readonly=0 (-1)
>>>>    qemu-system-x86_64: IOMMU_IOAS_MAP failed: Invalid argument
>>>>    qemu-system-x86_64: vfio_container_dma_map(0x55a21b03a150,
>>>> 0x380000004000, 0x4000, 0x7fce2c980000) = -22 (Invalid argument)
>>>>
>>>> This is because IOMMUFD reserved IOVAs areas are :
>>>>
>>>>   [ fee00000 - feefffff ]
>>>>   [ 8000000000 - ffffffffffffffff ] (39 bits address space)
>>>>
>>>> which were allocated when the device was initially attached.
>>>> The topology is basic. Something is wrong.
>>>
>>> Thanks for your report. This looks a hardware limit of
>>> host IOMMU address width(39) < guest physical address width.
>>>
>>> A similar issue with a fix submitted below, ccing related people.
>>> https://lists.gnu.org/archive/html/qemu-devel/2023-11/msg02937.html
>>> It looks the fix will not work for hotplug.
>>>
>>> Or below qemu cmdline may help:
>>> "-cpu host,host-phys-bits-limit=39"
>>
>> don't you have the same issue with legacy VFIO code, you should?
>
>I tend to be lazy and use seabios for guests on the command line.
>I do see the error with legacy VFIO and uefi.
>
>However, with the address space size work-around and iommufd, the
>error is different, an EFAULT now. Some page pinning issue it seems.

Yes, this reminds me of iommufd not supporting p2p mapping yet.
So EFAULT is expected. Maybe I should add a comment in 
docs/devel/vfio-iommufd.rst

Thanks
Zhenzhong