Re: [Qemu-devel] [PATCH v4 00/16] IOMMU: Enable interrupt remapping for Intel IOMMU

[Peter Xu]

On Tue, Apr 26, 2016 at 10:15:46AM +0200, Jan Kiszka wrote:
> On 2016-04-26 09:57, Jan Kiszka wrote:
> > On 2016-04-26 09:34, Peter Xu wrote:
> >> On Mon, Apr 25, 2016 at 09:24:12AM +0200, Jan Kiszka wrote:
> >>> On 2016-04-25 09:18, Peter Xu wrote:
> >>>> On Mon, Apr 25, 2016 at 07:16:19AM +0200, Jan Kiszka wrote:
> >>>>> On 2016-04-19 10:38, Peter Xu wrote:
> >>>>
> >>>> [...]
> >>>>
> >>>>>> By default, IR is disabled to be better compatible with current
> >>>>>> QEMU. To enable IR, we can using the following command to boot a
> >>>>>> IR-supported VM with virtio-net device with vhost (still do not
> >>>>>> support kvm-ioapic, so we need to specify kernel-irqchip={split|off}
> >>>>>> here):
> >>>>>>
> >>>>>> $ qemu-system-x86_64 -M q35,iommu=on,intr=on,kernel-irqchip=split \
> >>>>>
> >>>>> "intr" sounds a bit too much like "interrupt", not "interrupt
> >>>>> remapping". Why not use the kernel's form, "intremap"?
> >>>>
> >>>> Sure. It sounds nice to be aligned with the kernel one. Let me take
> >>>> it in v5.
> >>>>
> >>>>>
> >>>>>>      -enable-kvm -m 1024 \
> >>>>>>         -netdev tap,id=net0,vhost=on \
> >>>>>>      -device virtio-net-pci,netdev=user.0 \
> >>>>>>      -monitor telnet::3333,server,nowait \
> >>>>>>         /var/lib/libvirt/images/vm1.qcow2
> >>>>>>
> >>>>>> When guest boots, we can verify whether IR enabled by grepping the
> >>>>>> dmesg like:
> >>>>>>
> >>>>>> address@hidden ~]# journalctl -k | grep "DMAR-IR"
> >>>>>> Feb 19 11:21:23 localhost.localdomain kernel: DMAR-IR: IOAPIC id 0 
> >>>>>> under DRHD base  0xfed90000 IOMMU 0
> >>>>>> Feb 19 11:21:23 localhost.localdomain kernel: DMAR-IR: Enabled IRQ 
> >>>>>> remapping in xapic mode
> >>>>>>
> >>>>>> Currently supported devices:
> >>>>>>
> >>>>>> - Emulated/Splitted irqchip
> >>>>>> - Generic PCI Devices
> >>>>>> - vhost devices
> >>>>>> - pass through device support? Not tested, but suppose it should work.
> >>>>>
> >>>>> I've tested this series against my Jailhouse setup, and it works pretty
> >>>>> well! Actually considering to move my test setup over this branch.
> >>>>
> >>>> This is really encouraging feedback! Btw, thanks for all kinds of
> >>>> help on this patchset. :-)
> >>>>
> >>>>>
> >>>>> However, split irqchip still has some issues: When I boot a q35 machine
> >>>>> with Linux, the e1000 network adapter only gets a single IRQ delivered.
> >>>>> Interestingly, other IOAPIC IRQs like the keyboard work all the time. I
> >>>>> didn't debug this in details yet.
> >>>>
> >>>> I reproduced this problem. It seems that it fails even with
> >>>> kernel-irqchip=off. Will try to dig it out.
> >>>
> >>> Very good. Hope it can be easily fixed.
> >>
> >> Hi, Jan,
> >>
> >> The above issue should be caused by EOI missing of level-triggered
> >> interrupts. Before that, I was always using edge-triggered
> >> interrupts for test, so didn't encounter this one. Would you please
> >> help try below patch? It can be applied directly onto the series,
> >> and should solve the issue (it works on my test vm, and I'll take it
> >> in v5 as well if it also works for you):
> >>
> > 
> > Works here as well. I even made EIM working with some hack, though
> > Jailhouse spits out strange warnings, despite it works fine (x2apic
> > mode, split irqchip).
> 
> Corrections: the warnings are issued by qemu, not Jailhouse, e.g.
> 
> qemu-system-x86_64: VT-d Failed to remap interrupt for gsi 22.
> 
> I suspect that comes from the hand-over phase of Jailhouse, when it
> mutes all interrupts in the system while reconfiguring IR and IOAPIC.
> 
> Please convert this error (in kvm_arch_fixup_msi_route) into a trace
> point. It shall not annoy the host. Also check if you have more of such
> guest-triggerable error messages.

Okay. This should be the only one. I can use trace instead.

Meanwhile, I still suppose we should not seen it even with
error_report().. Would this happen when boot e.g. generic kernels?

-- peterx