Re: [Qemu-devel] [PATCH v2] msix: don't mask already masked vectors on reset

[Marcel Apfelbaum]

On 21/12/2017 17:08, Marcel Apfelbaum wrote:
> On 21/12/2017 16:01, Michael S. Tsirkin wrote:
> > On Fri, Dec 08, 2017 at 09:09:06AM -0500, Marcel Apfelbaum wrote:
> > > ----- Original Message -----
> > > > From: "Ladi Prosek" <address@hidden>
> > > > To: address@hidden
> > > > Cc: address@hidden, address@hidden, address@hidden, "alex williamson" 
> > > > <address@hidden>
> > > > Sent: Friday, December 8, 2017 8:31:13 AM
> > > > Subject: [PATCH v2] msix: don't mask already masked vectors on reset
> > > >
> > > > msix_mask_all() is supposed to invoke the release vector notifier if the
> > > > state of the
> > > > respective vector changed from unmasked to masked. The way it's currently
> > > > called from
> > > > msix_reset(), though, may result in calling the release notifier even if the
> > > > vector
> > > > is already masked.
> > > >
> > > > 1) msix_reset() clears out the msix_cap field and the msix_table.
> > > > 2) msix_mask_all() runs with was_masked=false for all vectors because of 1),
> > > > which
> > > >     results in calling the release notifier on all vectors.
> > > > 3) if msix_reset() is subsequently called again, it goes through the same
> > > > steps and
> > > >     calls the release notifier on all vectors again.
> > > >
> > > > This commit moves msix_mask_all() up so it runs before the device state is
> > > > lost. And
> > > > it adds an assignment to msix_function_masked so that the device remembers
> > > > that
> > > > MSI-X is masked.
> > > >
> > > > This is likely a low impact issue, found while debugging an already broken
> > > > device. It
> > > > is however easy to fix and the expectation that the use and release notifier
> > > > invocations
> > > > are always balanced is very natural.
> > > >
> > > > Signed-off-by: Ladi Prosek <address@hidden>
> > > > ---
> > > >
> > > > v1->v2:
> > > > * fixed typo in commit message "or" -> "to" (Marcel)
> > > > * directly set msix_function_masked to true instead of calling
> > > >    msix_update_function_masked() (Marcel)
> > > >
> > > >
> > > >   hw/pci/msix.c | 3 ++-
> > > >   1 file changed, 2 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/hw/pci/msix.c b/hw/pci/msix.c
> > > > index c944c02135..d6a4dbdb6b 100644
> > > > --- a/hw/pci/msix.c
> > > > +++ b/hw/pci/msix.c
> > > > @@ -500,11 +500,12 @@ void msix_reset(PCIDevice *dev)
> > > >           return;
> > > >       }
> > > >       msix_clear_all_vectors(dev);
> > > > +    msix_mask_all(dev, dev->msix_entries_nr);
> > > >       dev->config[dev->msix_cap + MSIX_CONTROL_OFFSET] &=
> > > >           ~dev->wmask[dev->msix_cap + MSIX_CONTROL_OFFSET];
> > > >       memset(dev->msix_table, 0, dev->msix_entries_nr * PCI_MSIX_ENTRY_SIZE);
> > > >       memset(dev->msix_pba, 0, QEMU_ALIGN_UP(dev->msix_entries_nr, 64) / 8);
> > > > -    msix_mask_all(dev, dev->msix_entries_nr);
> > > > +    dev->msix_function_masked = true;
> > > >   }
> > > >   /* PCI spec suggests that devices make it possible for software to configure
> > > > --
> > > > 2.13.6
> > >
> > > Thanks Ladi!
> > >
> > > Reviewed-by: Marcel Apfelbaum <address@hidden>
> >
> > This breaks make check though.
> >
> > Marcel - could you take a look please?
>
> Sure

We have a problem with the patch.

Moving msix_mask_all before resetting the msix_table to 0
modifies behavior since we don't set the PCI_MSIX_ENTRY_CTRL_MASKBIT
for each vector anymore (the information is lost).

What we need (I think) is to run msix_mask_all again,
this time without calling msix_handle_mask_update (only setting the bit).
(I tested it and the make check passes after it)

I think is risky to use the patch for now.
Thanks,
Marcel


> Thanks,
> Marcel