qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] Hitting 29 NIC limit


From: Anthony Liguori
Subject: Re: [Qemu-devel] Hitting 29 NIC limit
Date: Thu, 14 Oct 2010 07:57:20 -0500
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.12) Gecko/20100915 Lightning/1.0b1 Thunderbird/3.0.8

On 10/14/2010 07:10 AM, Daniel P. Berrange wrote:
On Thu, Oct 14, 2010 at 02:07:17PM +0200, Avi Kivity wrote:
  On 10/14/2010 12:54 AM, Anthony Liguori wrote:
On 10/13/2010 05:32 PM, Anjali Kulkarni wrote:
Hi,

Using the legacy way of starting up NICs, I am hitting a limitation
after 29
NICs ie no more than 29 are detected (that's because of the 32 PCI slot
limit on a single bus- 3 are already taken up)
I had initially increased the MAX_NICS to 48, just on my tree, to get to
more, but ofcource that wont work.
Is there any way to go beyond 29 NICs the legacy way?  What is the
maximum
that can be supported by the qdev mothod?
I got up to 104 without trying very hard using the following script:

args=""
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do
    args="$args -netdev user,id=eth${slot}_${fn}"
    args="$args -device
virtio-net-pci,addr=${slot}.${fn},netdev=eth${slot}_${fn},multifunction=on,romfile="
done
done

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args}
-enable-kvm

The key is to make the virtio-net devices multifunction and to fill
out all 8 functions for each slot.
This is unlikely to work right wrt pci hotplug.  If we want to support a
large number of interfaces, we need true multiport cards.
Or a PCI bridge to wire up more PCI buses, so we raise the max limit for
any type of device we emulate.

I've always been sceptical of this. When physical systems have a large number of NICs, it's via multiple functions, not a bunch of PCI bridges.

With just a handful of 8-port NICs, you can exceed the current slot-based limit on physical hardware. It's not an extremely common configuration but it does exist.

BTW, I don't think it's possible to hot-add physical functions. I believe I know of a card that supports dynamic add of physical functions (pre-dating SR-IOV).

Regards,

Anthony Liguori

Daniel




reply via email to

[Prev in Thread] Current Thread [Next in Thread]