Re: [PATCH v10 13/14] vfio-user: handle device interrupts

[John Johnson]

> On May 31, 2022, at 2:45 PM, Alex Williamson <alex.williamson@redhat.com> 
> wrote:
> 
> On Tue, 31 May 2022 22:03:14 +0100
> Stefan Hajnoczi <stefanha@gmail.com> wrote:
> 
>> On Tue, 31 May 2022 at 21:11, Alex Williamson
>> <alex.williamson@redhat.com> wrote:
>>> 
>>> On Tue, 31 May 2022 15:01:57 +0000
>>> Jag Raman <jag.raman@oracle.com> wrote:
>>> 
>>>>> On May 25, 2022, at 10:53 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>>>>> 
>>>>> On Tue, May 24, 2022 at 11:30:32AM -0400, Jagannathan Raman wrote:  
>>>>>> Forward remote device's interrupts to the guest
>>>>>> 
>>>>>> Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
>>>>>> Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
>>>>>> Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
>>>>>> ---
>>>>>> include/hw/pci/pci.h              |  13 ++++
>>>>>> include/hw/remote/vfio-user-obj.h |   6 ++
>>>>>> hw/pci/msi.c                      |  16 ++--
>>>>>> hw/pci/msix.c                     |  10 ++-
>>>>>> hw/pci/pci.c                      |  13 ++++
>>>>>> hw/remote/machine.c               |  14 +++-
>>>>>> hw/remote/vfio-user-obj.c         | 123 ++++++++++++++++++++++++++++++
>>>>>> stubs/vfio-user-obj.c             |   6 ++
>>>>>> MAINTAINERS                       |   1 +
>>>>>> hw/remote/trace-events            |   1 +
>>>>>> stubs/meson.build                 |   1 +
>>>>>> 11 files changed, 193 insertions(+), 11 deletions(-)
>>>>>> create mode 100644 include/hw/remote/vfio-user-obj.h
>>>>>> create mode 100644 stubs/vfio-user-obj.c  
>>>>> 
>>>>> It would be great if Michael Tsirkin and Alex Williamson would review
>>>>> this.  
>>>> 
>>>> Hi Michael and Alex,
>>>> 
>>>> Do you have any thoughts on this patch?  
>>> 
>>> Ultimately this is just how to insert callbacks to replace the default
>>> MSI/X triggers so you can send a vector# over the wire for a remote
>>> machine, right?  I'll let the code owners, Michael and Marcel, comment
>>> if they have grand vision how to architect this differently.  Thanks,  
>> 
>> An earlier version of the patch intercepted MSI-X at the msix_notify()
>> level, replacing the entire function. This patch replaces
>> msix_get_message() and msi_send_message(), leaving the masking logic
>> in place.
>> 
>> I haven't seen the latest vfio-user client implementation for QEMU,
>> but if the idea is to allow the guest to directly control the
>> vfio-user device's MSI-X table's mask bits, then I think this is a
>> different design from VFIO kernel where masking is emulated by QEMU
>> and not passed through to the PCI device.
> 
> Essentially what's happening here is an implementation of an interrupt
> handler callback in the remote QEMU instance.  The default handler is
> to simply write the MSI message data at the MSI message address of the
> vCPU, vfio-user replaces that with hijacking the MSI message itself to
> simply report the vector# so that the "handler", ie. trigger, can
> forward it to the client.  That's very analogous to the kernel
> implementation.
> 
> The equivalent masking we have today with vfio kernel would happen on
> the client side, where the MSI/X code might instead set a pending bit
> if the vector is masked on the client.  Likewise the possibility
> remains, just as it does on the kernel side, that the guest masking a
> vector could be relayed over ioctl/socket to set the equivalent mask on
> the host/remote.
> 
> I don't see a fundamental design difference here, but please point out
> if I'm missing something.  Thanks,
> 



        I don’t think you’ve missed anything.  We were trying to stay
close to the kernel implementation.

        Do you have any comments on the client side implementation I
sent on 5/5?

                                                        JJ