Re: [Qemu-devel] QEMU and vIOMMU support for emulated VF passthrough to nested (L2) VM

[Elijah Shakkour]

> -----Original Message-----
> From: Peter Xu <address@hidden>
> Sent: Monday, April 1, 2019 1:25 PM
> To: Elijah Shakkour <address@hidden>
> Cc: Knut Omang <address@hidden>; Michael S. Tsirkin
> <address@hidden>; Alex Williamson <address@hidden>;
> Marcel Apfelbaum <address@hidden>; Stefan Hajnoczi
> <address@hidden>; address@hidden
> Subject: Re: QEMU and vIOMMU support for emulated VF passthrough to
> nested (L2) VM
> 
> On Mon, Apr 01, 2019 at 09:12:38AM +0000, Elijah Shakkour wrote:
> >
> >
> > > -----Original Message-----
> > > From: Peter Xu <address@hidden>
> > > Sent: Monday, April 1, 2019 5:47 AM
> > > To: Elijah Shakkour <address@hidden>
> > > Cc: Knut Omang <address@hidden>; Michael S. Tsirkin
> > > <address@hidden>; Alex Williamson <address@hidden>;
> > > Marcel Apfelbaum <address@hidden>; Stefan Hajnoczi
> > > <address@hidden>; address@hidden
> > > Subject: Re: QEMU and vIOMMU support for emulated VF passthrough
> to
> > > nested (L2) VM
> > >
> > > On Sun, Mar 31, 2019 at 11:15:00AM +0000, Elijah Shakkour wrote:
> > >
> > > [...]
> > >
> > > > I didn't have DMA nor MMIO read/write working with my old command
> > > line.
> > > > But, when I removed all CPU flags and only provided "-cpu host", I
> > > > see that
> > > MMIO works.
> > > > Still, DMA read/write from emulated device doesn't work for VF.
> > > > For
> > > example:
> > > > Driver provides me a buffer pointer through MMIO write, this
> > > > address
> > > (pointer) is GPA of L2, and when I try to call pci_dma_read() with
> > > this address I get:
> > > > "
> > > > Unassigned mem read 0000000000000000 "
> > >
> > > I don't know where this error log was dumped but if it's during DMA
> > > then I agree it can probably be related to vIOMMU.
> > >
> >
> > This log is dumped from:
> > memory.c: unassigned_mem_read()
> >
> > > > As I said, my problem now is in translation of L2 GPA provided by
> > > > driver,
> > > when I call DMA read/write for this address from VF.
> > > > Any insights?
> > >
> > > I just noticed that you were using QEMU 2.12 [1].  If that's the
> > > case, please rebase to the latest QEMU, at least >=3.0 because
> > > there's major refactor of the shadow logic during 3.0 devel cycle AFAICT.
> > >
> >
> > Rebased to QEMU 3.1
> > Now I see the address I'm trying to read from in log but still same error:
> > "
> > Unassigned mem read 00000000f0481000
> > "
> > What do you suggest?
> 
> Would you please answer the questions that Knut asked?  Is it working for L1
> guest?  How about PF?

Both VF and PF are working for L1 guest.
I don't know how to passthrough PF to nested VM in hyper-v.
I don't invoke VF manually in hyper-v and pass it through to nested VM. I use 
hyper-v manager to configure and provide a VF for nested VM (I can see the VF 
only in the nested VM).

Did someone try to run emulated device in linux RH as nested L2 where L1 is 
windows hyper-v? Does DMA read/write work for this emulated device in this case?

> 
> You can also try to enable VT-d device log by appending:
> 
>   -trace enable="vtd_*"
> 
> In case it dumps anything useful for you.

Is there a way to open those traces to be dumped to stdout/stderr on the fly, 
instead of dtrace?

> 
> --
> Peter Xu