Re: [Qemu-devel] [PATCH] virtio-pci: enable bus master for old guests

[Greg Kurz]

On Wed, 10 Sep 2014 12:32:30 +0300
"Michael S. Tsirkin" <address@hidden> wrote:

> On Wed, Sep 10, 2014 at 01:44:49PM +0530, Nikunj A Dadhania wrote:
> > Greg Kurz <address@hidden> writes:
> > 
> > > On Mon, 8 Sep 2014 19:05:02 +0300
> > > "Michael S. Tsirkin" <address@hidden> wrote:
> > >
> > >> commit cc943c36faa192cd4b32af8fe5edb31894017d35
> > >>     pci: Use bus master address space for delivering MSI/MSI-X messages
> > >> breaks virtio-net for rhel6.[56] x86 guests because they don't
> > >> enable bus mastering for virtio PCI devices
> > >> 
> > >> Old guests forgot to enable bus mastering, enable it
> > >> automatically on DRIVER_OK.
> > >> 
> > >> Note: we should either back out the original patch from
> > >> stable or apply this one on top.
> > >> 
> > >> Cc: address@hidden
> > >> Reported-by: Greg Kurz <address@hidden>
> > >> Signed-off-by: Jan Kiszka <address@hidden>
> > >> Signed-off-by: Michael S. Tsirkin <address@hidden>
> > >> ---
> > >>  hw/virtio/virtio-pci.c | 2 ++
> > >>  1 file changed, 2 insertions(+)
> > >> 
> > >> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
> > >> index ddb5da1..af937d2 100644
> > >> --- a/hw/virtio/virtio-pci.c
> > >> +++ b/hw/virtio/virtio-pci.c
> > >> @@ -320,6 +320,8 @@ static void virtio_ioport_write(void *opaque, 
> > >> uint32_t addr, uint32_t val)
> > >>          if ((val & VIRTIO_CONFIG_S_DRIVER_OK) &&
> > >>              !(proxy->pci_dev.config[PCI_COMMAND] & PCI_COMMAND_MASTER)) 
> > >> {
> > >>              proxy->flags |= VIRTIO_PCI_FLAG_BUS_MASTER_BUG;
> > >> +            
> > >> memory_region_set_enabled(&proxy->pci_dev.bus_master_enable_region,
> > >> +                                      true);
> > >>          }
> > >>          break;
> > >>      case VIRTIO_MSI_CONFIG_VECTOR:
> > >
> > > Cc'ing Alexey for some SLOF and early boot of the ppc64 kernel expertise.
> > >
> > > Michael,
> > >
> > > This was enough to fix virtio-net in the rhel6.5 x86 guest case. 
> > > Unfortunately,
> > > this fails for rhel6.5 ppc64 because it is never called... 
> > 
> > > I did some debugging: it looks like the guest kernel calls the OF
> > > quisece call to flush pending DMA and disables bus master on the
> > > virtio-blk device (PCI_COMMAND == 0x3).
> > 
> > Getting confused, above you are talking about virtio-net and here it is
> > virtio-blk.
> > 
> > Anyways, the routines still remains same for both of them.  From SLOF
> > during init we set DRIVER_OK, and after using the device during the
> > quiesce, called from linux kernel VIRTIO_CONFIG_S_FAILED is set and then
> > a VIRTIO_DEVICE_RESET is done.
> 
> BTW, you really should start enabling bus mastering, avoid relying
> on the work-around we have for broken guests.
> 

FWIW during my debug session, I see that SLOF enables bus mastering...
unfortunately, it gets disabled at some point after the guest kernel
is started (around the ppc64 prom_init() call).

> > > The guest then continues to boot and hangs... It appears that waiting
> > > for the guest to issue VIRTIO_CONFIG_S_DRIVER_OK is not enough. Since
> > > we need this for MSI to work, I tried the following and it fixes the
> > > issue:
> > >
> > > diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
> > > index af937d2..3d72aa8 100644
> > > --- a/hw/virtio/virtio-pci.c
> > > +++ b/hw/virtio/virtio-pci.c
> > > @@ -111,9 +111,14 @@ static void virtio_pci_notify(DeviceState *d, 
> > > uint16_t vector)
> > >  {
> > >      VirtIOPCIProxy *proxy = to_virtio_pci_proxy_fast(d);
> > >  
> > > -    if (msix_enabled(&proxy->pci_dev))
> > > +    if (msix_enabled(&proxy->pci_dev)) {
> > > +        if (!(proxy->pci_dev.config[PCI_COMMAND] & PCI_COMMAND_MASTER)) {
> > > +            proxy->flags |= VIRTIO_PCI_FLAG_BUS_MASTER_BUG;
> > > +            
> > > memory_region_set_enabled(&proxy->pci_dev.bus_master_enable_region,
> > > +                                      true);
> > > +        }
> > >          msix_notify(&proxy->pci_dev, vector);
> > > -    else {
> > > +    } else {
> > >          VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
> > >          pci_set_irq(&proxy->pci_dev, vdev->isr & 1);
> > >      }
> > >
> > > If this is acceptable, I'll make it a helper and squash it into your 
> > > patch.
> > >
> > > Thoughts ?
> > >
> > > -- 
> > > Gregory Kurz                                     address@hidden
> > >                                                  address@hidden
> > > Software Engineer @ IBM/Meiosys                  http://www.ibm.com
> > > Tel +33 (0)562 165 496
> > >
> > > "Anarchy is about taking complete responsibility for yourself."
> > >         Alan Moore.
> 



-- 
Gregory Kurz                                     address@hidden
                                                 address@hidden
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.