Re: [Qemu-devel] Re: [RFC PATCH 4/7] ide: IOMMU support

[Chris Wright]

* Chris Wright (address@hidden) wrote:
> * Paul Brook (address@hidden) wrote:
> > > > The right approach IMHO is to convert devices to use bus-specific
> > > > functions to access memory.  The bus specific functions should have
> > > > a device argument as the first parameter.
> > > 
> > > As for ATS, the internal api to handle the device's dma reqeust needs
> > > a notion of a translated vs. an untranslated request.  IOW, if qemu ever
> > > had a device with ATS support, the device would use its local cache to
> > > translate the dma address and then submit a translated request to the
> > > pci bus (effectively doing a raw cpu physical memory* in that case).
> > 
> > Really? Can you provide an documentation to support this claim?

Wow...color me surprised...there's actually some apparently public
"training" docs that might help give a more complete view:

http://www.pcisig.com/developers/main/training_materials/get_document?doc_id=0ab681ba7001e40cdb297ddaf279a8de82a7dc40

ATS discussion starts on slide 23.

> > My impression is that there is no difference between translated and 
> > untranslated devices, and the translation is explicitly disabled by 
> > software.

And now that I re-read that sentence, I see what you are talking about.
Yes, there is the above notion as well.

A device can live in a 1:1 mapping of device address space to physical
memory.  This could be achieved in a few ways (all done by the OS software
programming the IOMMU).

One is to simply create a set of page tables that map 1:1 all of device
memory to physical memory.  Another is to somehow mark the device as
special (either omit translation tables or mark the translation entry
as effectively "do not translate").  This is often referred to as Pass
Through mode.  But this is not the same as ATS.

Pass Through mode is the functional equivalent of disabling the
translation/isolation capabilities of the IOMMU.  It's typically used
when an OS wants to keep a device for itself and isn't interested in
the isolation properties of the IOMMU.  It then only creates isolating
translation tables for devices it's giving to unprivileged software
(e.g. Linux/KVM giving a device to a guest, Linux giving a device to
user space process, etc.)

> ATS allows an I/O device to request a translation from the IOMMU.
> The device can then cache that translation and use the translated address
> in a PCIe memory transaction.  PCIe uses a couple of previously reserved
> bits in the transaction layer packet header to describe the address
> type for memory transactions.  The default (00) maps to legacy PCIe and
> describes the memory address as untranslated.  This is the normal mode,
> and could then incur a translation if an IOMMU is present and programmed
> w/ page tables, etc. as is passes through the host bridge.
> 
> Another type is simply a transaction requesting a translation.  This is
> new, and allows a device to request (and cache) a translation from the
> IOMMU for subsequent use.
> 
> The third type is a memory transaction tagged as already translated.
> This is the type of transaction an ATS capable I/O device will generate
> when it was able to translate the memory address from its own cache.
> 
> Of course, there's also an invalidation request that the IOMMU can send
> to ATS capable I/O devices to invalidate the cached translation.

thanks,
-chris