Re: [Qemu-devel] [PATCH v3 3/3] QEMU-AER: Qemu changes to support AER for VFIO-PCI devices

[Gleb Natapov]

On Tue, Feb 05, 2013 at 06:37:35AM -0700, Alex Williamson wrote:
> On Tue, 2013-02-05 at 12:05 +0000, Pandarathil, Vijaymohan R wrote:
> > 
> > > -----Original Message-----
> > > From: address@hidden [mailto:linux-pci-
> > > address@hidden On Behalf Of Gleb Natapov
> > > Sent: Tuesday, February 05, 2013 3:37 AM
> > > To: Pandarathil, Vijaymohan R
> > > Cc: Blue Swirl; Alex Williamson; Bjorn Helgaas; Ortiz, Lance E;
> > > address@hidden; address@hidden; address@hidden;
> > > address@hidden
> > > Subject: Re: [PATCH v3 3/3] QEMU-AER: Qemu changes to support AER for 
> > > VFIO-
> > > PCI devices
> > > 
> > > On Tue, Feb 05, 2013 at 10:59:41AM +0000, Pandarathil, Vijaymohan R wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Gleb Natapov [mailto:address@hidden
> > > > > Sent: Tuesday, February 05, 2013 1:21 AM
> > > > > To: Pandarathil, Vijaymohan R
> > > > > Cc: Blue Swirl; Alex Williamson; Bjorn Helgaas; Ortiz, Lance E;
> > > > > address@hidden; address@hidden; address@hidden;
> > > > > address@hidden
> > > > > Subject: Re: [PATCH v3 3/3] QEMU-AER: Qemu changes to support AER for
> > > VFIO-
> > > > > PCI devices
> > > > >
> > > > > On Tue, Feb 05, 2013 at 09:05:19AM +0000, Pandarathil, Vijaymohan R
> > > wrote:
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Gleb Natapov [mailto:address@hidden
> > > > > > > Sent: Tuesday, February 05, 2013 12:05 AM
> > > > > > > To: Blue Swirl
> > > > > > > Cc: Pandarathil, Vijaymohan R; Alex Williamson; Bjorn Helgaas;
> > > Ortiz,
> > > > > Lance
> > > > > > > E; address@hidden; address@hidden; linux-
> > > > > address@hidden;
> > > > > > > address@hidden
> > > > > > > Subject: Re: [PATCH v3 3/3] QEMU-AER: Qemu changes to support AER
> > > for
> > > > > VFIO-
> > > > > > > PCI devices
> > > > > > >
> > > > > > > On Sun, Feb 03, 2013 at 04:36:11PM +0000, Blue Swirl wrote:
> > > > > > > > On Sun, Feb 3, 2013 at 2:10 PM, Pandarathil, Vijaymohan R
> > > > > > > > <address@hidden> wrote:
> > > > > > > > >         - Create eventfd per vfio device assigned to a guest
> > > and
> > > > > > > register an
> > > > > > > > >           event handler
> > > > > > > > >
> > > > > > > > >         - This fd is passed to the vfio_pci driver through the
> > > > > SET_IRQ
> > > > > > > ioctl
> > > > > > > > >
> > > > > > > > >         - When the device encounters an error, the eventfd is
> > > > > signalled
> > > > > > > > >           and the qemu eventfd handler gets invoked.
> > > > > > > > >
> > > > > > > > >         - In the handler decide what action to take. Current
> > > action
> > > > > > > taken
> > > > > > > > >           is to terminate the guest.
> > > > > > > >
> > > > > > > > Usually this is not OK, but I guess this is not guest
> > > triggerable.
> > > > > > > >
> > > > > > > Still not OK. Why not stop a guest with appropriate stop reason?
> > > > > >
> > > > > > The thinking was that since this is a hardware error, we would want
> > > to
> > > > > stop the guest at the earliest. The hw_error() routine which aborts 
> > > > > the
> > > > > qemu process was suggested by Alex and that seemed appropriate. 
> > > > > Earlier
> > > I
> > > > > was using qemu_system_shutdown_request().  Any suggestions ?
> > > > > >
> > > > > I am thinking vm_stop(). Stopping SMP guest (and UP too in fact)
> > > > > involves sending IPIs to other cpus running guest's vcpus. Both exit()
> > > > > and vm_stop() will do it, but former is implicitly in the kernel and
> > > > > later is explicitly in QEMU.
> > > >
> > > > I had used vm_stop(RUN_STATE_SHUTDOWN) earlier in my code. But while
> > > testing, guest ended up in a hang rather than exiting. There seems to some
> > > cleanup work which is being done as part of vm_stop. In our case, we 
> > > wanted
> > > the guest to exit immediately. So use of hw_error() seemed appropriate.
> > > >
> > > What makes you think it hang? It stopped, precisely what it should do if
> > > you call vm_stop(). Now it is possible for vm user to investigate what
> > > happened and even salvage some data from guest memory.
> > 
> > That was ignorance on my part on the expected behavior of vm_stop(). 
> > So what you are suggesting is to stop the guest displaying an appropriate 
> > error/next-steps message and have the users do any 
> > data-collection/investigation 
> > and then manually kill the guest, if they so desire. Right ?
> > 
> > Sounds reasonable. As long as the guest is not touching the device, it 
> > should be okay.
> > Alex, Any comments ?
> 
> What's the libvirt behavior when a guest goes to vm_stop?  My only
> concern would be whether the user is going to be confused by a state
> where the vm is still up, but not running.  I imagine they'll have to
> manually stop it and restart it to continue.  Thanks,
> 
vm_stop() is already the behaviour on KVM internal errors, so management
stack knows how to handle those. Of course vfio should use meaningful
stop reason and not just reuse existing one.

--
                        Gleb.