Re: [Qemu-devel] [PATCH V4 00/19] Support more virtio queues

[Michael S. Tsirkin]

On Thu, Mar 19, 2015 at 01:24:53PM +0800, Jason Wang wrote:
> 
> 
> On Wed, Mar 18, 2015 at 8:58 PM, Michael S. Tsirkin <address@hidden> wrote:
> >On Wed, Mar 18, 2015 at 05:34:50PM +0800, Jason Wang wrote:
> >> We current limit the max virtio queues to 64. This is not sufficient
> >> to support multiqueue devices (e.g recent Linux support up to 256
> >> tap queues). So this series tries to let virtio to support more queues.
> >> No much works need to be done except:
> >> - Introducing transport specific queue limitation.
> >> - Let each virtio transport to use specific limit.
> >> - Speedup the MSI-X masking and unmasking through per vector queue
> >>   list, and increase the maximum MSI-X vectors supported by qemu.
> >> - With the above enhancements, increase the maximum number of
> >>   virtqueues supported by PCI from 64 to 513.
> >> - Compat the changes for legacy machine types.
> >
> >What are the compatibility considerations here?
> 
> Two considerations:
> 1) To keep msix bar size to 4K for legacy machine types
> 2) Limit the pci queue max to 64 for legacy machine types

2 seems not relevant to me.

> >
> >
> >> With this patch, we can support up to 256 queues. Since x86 can only
> >> allow about 240 interrupt vectors for MSI-X, current Linux driver can
> >> only have about 80 queue pairs has their private MSI-X interrupt
> >> vectors. With sharing IRQ with queue pairs (RFC posted in
> >> https://lkml.org/lkml/2014/12/25/169), Linux driver can have up
> >> to about 186 queue pairs has their private MSI-X interrupt vectors.
> >> Stress/migration test on virtio-pci, compile test on other
> >> targets. And make check on s390x-softmmu and ppc64-softmmu.
> >> Cc: Paolo Bonzini <address@hidden>
> >> Cc: Richard Henderson <address@hidden>
> >> Cc: Michael S. Tsirkin <address@hidden>
> >> Cc: Alexander Graf <address@hidden>
> >> Cc: Keith Busch <address@hidden>
> >> Cc: Kevin Wolf <address@hidden>
> >> Cc: Stefan Hajnoczi <address@hidden>
> >> Cc: Christian Borntraeger <address@hidden>
> >> Cc: Cornelia Huck <address@hidden>
> >> Cc: Amit Shah <address@hidden>
> >> Cc: address@hidden
> >> Please review
> >> Thanks
> >> Changes from V3:
> >> - rebase to master and target to 2.4
> >> - handling compat issues for spapr
> >> - fixes for hmp command completion when we have a nic with 256 queues
> >> - using VIRTIO_NO_VECTOR instead of 0 for invalid virtqueue in
> >>   virtio-ccw
> >> - fix compile issues for ppc64-softmmu
> >> - don't export VIRTIO_CCW_QUEUE_MAX by introducing a gsi limit and
> >>   inherit in ccw
> >> - use transport specific queue limit in virtio-serial
> >> - correct the stale comment for AdapterRoutes and move it to ccw patch
> >> - replace 128 with queue_max + 64 and add a comment for this in
> >>   virtio_ccw_notify()
> >> Changes from V2:
> >> - move transport specific limitation to their implementation. (The
> >>   left is VIRTIO_CCW_QUEUE_MAX since I fail to find a common header
> >>   files other than virtio.h)
> >> - use virtio_get_queue_max() if possible in virtio.c
> >> - AdapterRoutes should use ccw limit
> >> - introduce a vector to queue mapping for virito devices and speedup
> >>   the MSI-X masking and unmasking through this.
> >> Changes from V1:
> >> - add a validation against the bus limitation
> >> - switch to use a bus specific queue limit instead of a global one,
> >>   this will allow us to just increase the limit of one transport
> >>   without disturbing others.
> >> - only increase the queue limit of virtio-pci
> >> - limit the maximum number of virtio queues to 64 for legacy machine
> >>   types
> >> Jason Wang (19):
> >>   pc: add 2.4 machine types
> >>   spapr: add machine type specific instance init function
> >>   ppc: spapr: add 2.4 machine type
> >>   monitor: replace the magic number 255 with MAX_QUEUE_NUM
> >>   monitor: check return value of qemu_find_net_clients_except()
> >>   virtio-ccw: using VIRTIO_NO_VECTOR instead of 0 for invalid virtqueue
> >>   virtio-net: validate backend queue numbers against bus limitation
> >>   virtio-net: fix the upper bound when trying to delete queues
> >>   virito: introduce bus specific queue limit
> >>   virtio-ccw: introduce ccw specific queue limit
> >>   virtio-s390: switch to bus specific queue limit
> >>   virtio-mmio: switch to bus specific queue limit
> >>   virtio-pci: switch to use bus specific queue limit
> >>   virtio: introduce vector to virtqueues mapping
> >>   virtio: introduce virtio_queue_get_index()
> >>   virtio-pci: speedup MSI-X masking and unmasking
> >>   virtio-pci: increase the maximum number of virtqueues to 513
> >>   pci: remove hard-coded bar size in msix_init_exclusive_bar()
> >>   virtio-pci: introduce auto_msix_bar_size property
> >>  hw/block/nvme.c                |  2 +-
> >>  hw/char/virtio-serial-bus.c    |  2 +-
> >>  hw/i386/pc_piix.c              | 42 ++++++++++++++++++++---
> >>  hw/i386/pc_q35.c               | 39 +++++++++++++++++++--
> >>  hw/misc/ivshmem.c              |  2 +-
> >>  hw/net/virtio-net.c            |  9 ++++-
> >>  hw/pci/msix.c                  | 18 ++++------
> >>  hw/ppc/spapr.c                 | 70
> >>+++++++++++++++++++++++++++++++++++--
> >>  hw/s390x/s390-virtio-bus.c     |  7 ++--
> >>  hw/s390x/s390-virtio-ccw.c     |  7 ++--
> >>  hw/s390x/virtio-ccw.c          | 20 +++++++----
> >>  hw/scsi/virtio-scsi.c          |  4 +--
> >>  hw/virtio/virtio-mmio.c        |  7 ++--
> >>  hw/virtio/virtio-pci.c         | 78
> >>++++++++++++++++++++++++++++--------------
> >>  hw/virtio/virtio-pci.h         |  3 ++
> >>  hw/virtio/virtio.c             | 75
> >>+++++++++++++++++++++++++++++++---------
> >>  include/hw/compat.h            | 11 ++++++
> >>  include/hw/pci/msix.h          |  2 +-
> >>  include/hw/s390x/s390_flic.h   |  4 ++-
> >>  include/hw/virtio/virtio-bus.h |  2 ++
> >>  include/hw/virtio/virtio.h     |  7 ++--
> >>  monitor.c                      | 25 ++++++++------
> >>  22 files changed, 339 insertions(+), 97 deletions(-)
> >>   --  2.1.0