Re: [Qemu-devel] [PATCH 00/12] Multiqueue virtio-net

[Jason Wang]

On 01/15/2013 03:44 AM, Anthony Liguori wrote:
> Jason Wang <address@hidden> writes:
>
>> Hello all:
>>
>> This seires is an update of last version of multiqueue virtio-net support.
>>
>> Recently, linux tap gets multiqueue support. This series implements basic
>> support for multiqueue tap, nic and vhost. Then use it as an infrastructure 
>> to
>> enable the multiqueue support for virtio-net.
>>
>> Both vhost and userspace multiqueue were implemented for virtio-net, but
>> userspace could be get much benefits since dataplane like parallized 
>> mechanism
>> were not implemented.
>>
>> User could start a multiqueue virtio-net card through adding a "queues"
>> parameter to tap.
>>
>> ./qemu -netdev tap,id=hn0,queues=2,vhost=on -device virtio-net-pci,netdev=hn0
>>
>> Management tools such as libvirt can pass multiple pre-created fds through
>>
>> ./qemu -netdev tap,id=hn0,queues=2,fd=X,fd=Y -device
>> virtio-net-pci,netdev=hn0
> I'm confused/frightened that this syntax works.  You shouldn't be
> allowed to have two values for the same property.  Better to have a
> syntax like fd[0]=X,fd[1]=Y or something along those lines.

Yes, but this what current a StringList type works for command line.
Some other parameters such as dnssearch, hostfwd and guestfwd have
already worked in this way. Looks like your suggestions need some
extension on QemuOps visitor, maybe we can do this on top.

Thanks
>
> Regards,
>
> Anthony Liguori
>
>> You can fetch and try the code from:
>> git://github.com/jasowang/qemu.git
>>
>> Patch 1 adds a generic method of creating multiqueue taps and implement the
>> linux part.
>> Patch 2 - 4 introduce some helpers which could be used to refactor the nic
>> emulation codes to support multiqueue.
>> Patch 5 introduces multiqueue support for qemu networking code: each peers of
>> NetClientState were abstracted as a queue. Though this, most of the codes 
>> could
>> be reusued without change.
>> Patch 6 adds basic multiqueue support for vhost which could let vhost just
>> handle a subset of all virtqueues.
>> Patch 7-8 introduce new helpers of virtio which is needed by multiqueue
>> virtio-net.
>> Patch 9-12 implement the multiqueue support of virtio-net
>>
>> Changes from RFC v2:
>> - rebase the codes to latest qemu
>> - align the multiqueue virtio-net implementation to virtio spec
>> - split the patches into more smaller patches
>> - set_link and hotplug support
>>
>> Changes from RFC V1:
>> - rebase to the latest
>> - fix memory leak in parse_netdev
>> - fix guest notifiers assignment/de-assignment
>> - changes the command lines to:
>>    qemu -netdev tap,queues=2 -device virtio-net-pci,queues=2
>>
>> Reference:
>> v2: http://lists.gnu.org/archive/html/qemu-devel/2012-06/msg04108.html
>> v1: http://comments.gmane.org/gmane.comp.emulators.qemu/100481
>>
>> Perf Numbers:
>>
>> Two Intel Xeon 5620 with direct connected intel 82599EB
>> Host/Guest kernel: David net tree
>> vhost enabled
>>
>> - lots of improvents of both latency and cpu utilization in request-reponse 
>> test
>> - get regression of guest sending small packets which because TCP tends to 
>> batch
>>   less when the latency were improved
>>
>> 1q/2q/4q
>> TCP_RR
>>  size #sessions trans.rate  norm trans.rate  norm trans.rate  norm
>> 1 1     9393.26   595.64  9408.18   597.34  9375.19   584.12
>> 1 20    72162.1   2214.24 129880.22 2456.13 196949.81 2298.13
>> 1 50    107513.38 2653.99 139721.93 2490.58 259713.82 2873.57
>> 1 100   126734.63 2676.54 145553.5  2406.63 265252.68 2943
>> 64 1    9453.42   632.33  9371.37   616.13  9338.19   615.97
>> 64 20   70620.03  2093.68 125155.75 2409.15 191239.91 2253.32
>> 64 50   106966    2448.29 146518.67 2514.47 242134.07 2720.91
>> 64 100  117046.35 2394.56 190153.09 2696.82 238881.29 2704.41
>> 256 1   8733.29   736.36  8701.07   680.83  8608.92   530.1
>> 256 20  69279.89  2274.45 115103.07 2299.76 144555.16 1963.53
>> 256 50  97676.02  2296.09 150719.57 2522.92 254510.5  3028.44
>> 256 100 150221.55 2949.56 197569.3  2790.92 300695.78 3494.83
>> TCP_CRR
>>  size #sessions trans.rate  norm trans.rate  norm trans.rate  norm
>> 1 1     2848.37  163.41 2230.39  130.89 2013.09  120.47
>> 1 20    23434.5  562.11 31057.43 531.07 49488.28 564.41
>> 1 50    28514.88 582.17 40494.23 605.92 60113.35 654.97
>> 1 100   28827.22 584.73 48813.25 661.6  61783.62 676.56
>> 64 1    2780.08  159.4  2201.07  127.96 2006.8   117.63
>> 64 20   23318.51 564.47 30982.44 530.24 49734.95 566.13
>> 64 50   28585.72 582.54 40576.7  610.08 60167.89 656.56
>> 64 100  28747.37 584.17 49081.87 667.87 60612.94 662
>> 256 1   2772.08  160.51 2231.84  131.05 2003.62  113.45
>> 256 20  23086.35 559.8  30929.09 528.16 48454.9  555.22
>> 256 50  28354.7  579.85 40578.31 607    60261.71 657.87
>> 256 100 28844.55 585.67 48541.86 659.08 61941.07 676.72
>> TCP_STREAM guest receiving
>>  size #sessions throughput  norm throughput  norm throughput  norm
>> 1 1     16.27   1.33   16.1    1.12   16.13   0.99
>> 1 2     33.04   2.08   32.96   2.19   32.75   1.98
>> 1 4     66.62   6.83   68.3    5.56   66.14   2.65
>> 64 1    896.55  56.67  914.02  58.14  898.9   61.56
>> 64 2    1830.46 91.02  1812.02 64.59  1835.57 66.26
>> 64 4    3626.61 142.55 3636.25 100.64 3607.46 75.03
>> 256 1   2619.49 131.23 2543.19 129.03 2618.69 132.39
>> 256 2   5136.58 203.02 5163.31 141.11 5236.51 149.4
>> 256 4   7063.99 242.83 9365.4  208.49 9421.03 159.94
>> 512 1   3592.43 165.24 3603.12 167.19 3552.5  169.57
>> 512 2   7042.62 246.59 7068.46 180.87 7258.52 186.3
>> 512 4   6996.08 241.49 9298.34 206.12 9418.52 159.33
>> 1024 1  4339.54 192.95 4370.2  191.92 4211.72 192.49
>> 1024 2  7439.45 254.77 9403.99 215.24 9120.82 222.67
>> 1024 4  7953.86 272.11 9403.87 208.23 9366.98 159.49
>> 4096 1  7696.28 272.04 7611.41 270.38 7778.71 267.76
>> 4096 2  7530.35 261.1  8905.43 246.27 8990.18 267.57
>> 4096 4  7121.6  247.02 9411.75 206.71 9654.96 184.67
>> 16384 1 7795.73 268.54 7780.94 267.2  7634.26 260.73
>> 16384 2 7436.57 255.81 9381.86 220.85 9392    220.36
>> 16384 4 7199.07 247.81 9420.96 205.87 9373.69 159.57
>> TCP_MAERTS guest sending
>>  size #sessions throughput  norm throughput  norm throughput  norm
>> 1 1     15.94   0.62   15.55   0.61   15.13   0.59
>> 1 2     36.11   0.83   32.46   0.69   32.28   0.69
>> 1 4     71.59   1      68.91   0.94   61.52   0.77
>> 64 1    630.71  22.52  622.11  22.35  605.09  21.84
>> 64 2    1442.36 30.57  1292.15 25.82  1282.67 25.55
>> 64 4    3186.79 42.59  2844.96 36.03  2529.69 30.06
>> 256 1   1760.96 58.07  1738.44 57.43  1695.99 56.19
>> 256 2   4834.23 95.19  3524.85 64.21  3511.94 64.45
>> 256 4   9324.63 145.74 8956.49 116.39 6720.17 73.86
>> 512 1   2678.03 84.1   2630.68 82.93  2636.54 82.57
>> 512 2   9368.17 195.61 9408.82 204.53 5316.3  92.99
>> 512 4   9186.34 209.68 9358.72 183.82 9489.29 160.42
>> 1024 1  3620.71 109.88 3625.54 109.83 3606.61 112.35
>> 1024 2  9429    258.32 7082.79 120.55 7403.53 134.78
>> 1024 4  9430.66 290.44 9499.29 232.31 9414.6  190.92
>> 4096 1  9339.28 296.48 9374.23 372.88 9348.76 298.49
>> 4096 2  9410.53 378.69 9412.61 286.18 9409.75 278.31
>> 4096 4  9487.35 374.1  9556.91 288.81 9441.94 221.64
>> 16384 1 9380.43 403.8  9379.78 399.13 9382.42 393.55
>> 16384 2 9367.69 406.93 9415.04 312.68 9409.29 300.9
>> 16384 4 9391.96 405.17 9695.12 310.54 9423.76 223.47
>>
>>
>> Jason Wang (12):
>>   tap: multiqueue support
>>   net: introduce qemu_get_queue()
>>   net: introduce qemu_get_nic()
>>   net: intorduce qemu_del_nic()
>>   net: multiqueue support
>>   vhost: multiqueue support
>>   virtio: introduce virtio_queue_del()
>>   virtio: add a queue_index to VirtQueue
>>   virtio-net: separate virtqueue from VirtIONet
>>   virtio-net: multiqueue support
>>   virtio-net: migration support for multiqueue
>>   virtio-net: compat multiqueue support
>>
>>  hw/cadence_gem.c        |   16 +-
>>  hw/dp8393x.c            |   16 +-
>>  hw/e1000.c              |   28 ++--
>>  hw/eepro100.c           |   18 +-
>>  hw/etraxfs_eth.c        |   10 +-
>>  hw/lan9118.c            |   16 +-
>>  hw/lance.c              |    2 +-
>>  hw/mcf_fec.c            |   12 +-
>>  hw/milkymist-minimac2.c |   10 +-
>>  hw/mipsnet.c            |   10 +-
>>  hw/musicpal.c           |    6 +-
>>  hw/ne2000-isa.c         |    4 +-
>>  hw/ne2000.c             |   12 +-
>>  hw/opencores_eth.c      |   12 +-
>>  hw/pc_piix.c            |    4 +
>>  hw/pcnet-pci.c          |    4 +-
>>  hw/pcnet.c              |   12 +-
>>  hw/qdev-properties.c    |   46 ++++-
>>  hw/qdev-properties.h    |    6 +-
>>  hw/rtl8139.c            |   20 +-
>>  hw/smc91c111.c          |   10 +-
>>  hw/spapr_llan.c         |    8 +-
>>  hw/stellaris_enet.c     |   10 +-
>>  hw/usb/dev-network.c    |   16 +-
>>  hw/vhost.c              |   52 +++--
>>  hw/vhost.h              |    2 +
>>  hw/vhost_net.c          |    7 +-
>>  hw/vhost_net.h          |    2 +-
>>  hw/virtio-net.c         |  523 
>> ++++++++++++++++++++++++++++++++++-------------
>>  hw/virtio-net.h         |   27 +++-
>>  hw/virtio.c             |   17 ++
>>  hw/virtio.h             |    3 +
>>  hw/xen_nic.c            |   14 +-
>>  hw/xgmac.c              |   10 +-
>>  hw/xilinx_axienet.c     |   10 +-
>>  hw/xilinx_ethlite.c     |   10 +-
>>  net.c                   |  198 ++++++++++++++----
>>  net.h                   |   31 +++-
>>  net/tap-aix.c           |   18 ++-
>>  net/tap-bsd.c           |   18 ++-
>>  net/tap-haiku.c         |   18 ++-
>>  net/tap-linux.c         |   70 ++++++-
>>  net/tap-linux.h         |    4 +
>>  net/tap-solaris.c       |   18 ++-
>>  net/tap-win32.c         |   10 +
>>  net/tap.c               |  248 ++++++++++++++++-------
>>  net/tap.h               |    8 +-
>>  qapi-schema.json        |    5 +-
>>  savevm.c                |    2 +-
>>  49 files changed, 1177 insertions(+), 456 deletions(-)
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to address@hidden
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html