Re: [Qemu-devel] [PATCH] vfio: Align iova also to IOMMU page size

[Pavel Fedin]

 Hello!

> >  My device defines this BAR to be of 2M size. In this case qemu splits it 
> > up into three
> > regions:
> >  1) Region below the MSI-X table (it's called "mmap", for me it's empty 
> > because table offset
> > is 0)
> >  2) MSI-X table itself (20 vectors = 0x00000140 bytes for me).
> >  3) Region above the MSi-X table (it's called "mmap msix-hi"). Size for my 
> > case is 2M -
> > REAL_HOST_PAGE_ALIGN(0x140) = 0x1FF000.
> > Regions (1) and (3) are passed through and directly mapped by VFIO, region 
> > (2) is emulated.
> >  Now, we have PBA. The device says that PBA is placed in memory specified 
> > by the same BAR as
> > table, and its offset is 0x000f0000. PBA is also emulated by qemu, and as a 
> > result it overlaps
> > "mmap msix-hi" region, which breaks up into two fragments as a consequence:
> >  3a) offset 0x00000 size 0x0EF000
> >  3b) offset 0xEF008 size 0x10FFF8
> >  PBA sits between these fragments, having only 8 bytes in size.
> >  Attempt to map (3b) causes the problem. vfio_mmap_region() is expected to 
> > align this up,
> > and it does, but it does it to a minimum possible granularity for ARM 
> > platform, which, as qemu
> > thinks, is always 1K.
> 
> Yep, on x86 we'd have target page size == host page size == minimum
> iommu page size and we'd align that up to something that can be mapped,
> which means we wouldn't have an iommu mapping for peer-to-peer in this
> space, but nobody is likely to notice.

 So, OK, to keep the quoting short... I've also read your previous reply about 
that "rounding up just works". Let's sum things up.

1. Rounding up "just works" in my case too. So i don't see a direct reason to 
change it.
2. The only problem is rounding up to 1K, which TARGET_PAGE_ALIGN() does on 
ARM. What we need is 4K,
   which is appropriate for host too. And, by the way, for modern ARM guests 
too. So, we can do either of:
   a) Keep my original approach - TARGET_PAGE_ALIGN(), then align to 
vfio_container_granularity().
   b) Just align to vfio_container_granularity() instead of using 
TARGET_PAGE_ALIGN().
   c) Use HOST_PAGE_ALIGN() instead of TARGET_PAGE_ALIGN() (what Peter 
suggested).

   On x86 there would be no difference, because all these alignments are 
identical. On ARM, actually, all of these approaches would also give correct 
result, because it's only TARGET_PAGE_ALIGN() wrong in this case. So - what do 
you think is the most appropriate? Let's make a choice and i'll send a proper 
patch.

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia