Re: [RFC PATCH v2 20/22] i386/xen: HVMOP_set_param / HVM_PARAM_CALLBACK_IRQ

[David Woodhouse]

On Tue, 2022-12-20 at 17:29 +0000, David Woodhouse wrote:
> On 20 December 2022 17:25:31 GMT, Paul Durrant <xadimgnik@gmail.com> wrote:
> > On 20/12/2022 16:27, David Woodhouse wrote:
> > > 
> > > 
> > > On 20 December 2022 13:56:39 GMT, Paul Durrant
> > > <xadimgnik@gmail.com> wrote:
> > > > The callback via is essentially just another line interrupt but
> > > > its assertion is closely coupled with the vcpu upcall_pending
> > > > bit. Because that is being dealt with in-kernel then the via
> > > > should be dealt with in-kernel, right?
> > > 
> > > Not right now, because there's not a lot of point in kernel
> > > acceleration in the case that it's delivered as GSI. There's no
> > > per-vCPU delivery, so it doesn't get used for IPI, for timers.
> > > None of the stuff we want to accelerate in-kernel. Only the
> > > actual PV drivers.
> > > 
> > > If we do FIFO event channels in the future then the kernel will
> > > probably need to own those; they need spinlocks and you can have
> > > *both* userspace and kernel delivering with test-and-set-bit
> > > sequences like you can with 2level.
> > > 
> > > Even so, I was tempted to add a VFIO-like eventfd pair for the
> > > vCPU0 evtchn_upcall_pending and let the kernel drive it... but
> > > qemu doesn't even do the EOI side of that as $DEITY intended, so
> > > I didn't.
> > 
> > My point was that clearing upcall_pending is essentially the
> > equivalent of EOI-at-device i.e. it's the thing that drops the
> > line. So, without some form of interception, we need some way to
> > check it at an appropriate time... and as you say, there may be no
> > exit to VMM for EOI of the APIC. So when?
> 
> If we have an in-kernel APIC then it has a notifier callback on EOI
> and we can poll evtchn_upcall_pending then. That's another point in
> favour of handling it in kernel.
> 
> And for userspace APIC we *do* get an exit of course... but QEMU
> lacks that notifier mechanism for EOI of pseudo-level interrupts for
> that "should it still be pending?" check. 

So, I now have the qemu/backend side of event channels hooked up, and a
basic xenstore implementation that just returns ENOSYS to everything
for now. As before, at
https://git.infradead.org/users/dwmw2/qemu.git/shortlog/refs/heads/xenfv

I played with booting the guest with 'xen_no_vector_callback' on its
command line.

With kernel-irqchip=split (i.e. PIC/APIC in userspace) that seems to be
working OK. We *do* take that exit to qemu, and qemu spots that it
needs to deassert the GSI.

With the *kernel* PIC/APIC, we get a brief IRQ storm on the PCI INTX
every time:

[root@localhost ~]# grep xen /proc/interrupts 
 10:      80001          0   IO-APIC  10-fasteoi   xen-platform-pci
 24:          8          0   xen-dyn    -event     xenbus

In fact, unless CPU0 takes an exit, that storm might not be very brief
at all.

I *could* fix this up in the kernel via kvm_notify_acked_irq(), but I
suspect the better answer is just to declare that the Xen support (or
at least GSI evtchn delivery) requires split irqchip.

smime.p7s
Description: S/MIME cryptographic signature