[Qemu-devel] increasing number of vhost devices

[Antonin Bas]

Hi all,

In the most versions of the linux kernel (3.9+), kvm has increased the
number of user memory slots to 125:

#define KVM_SOFT_MAX_VCPUS 160
#define KVM_USER_MEM_SLOTS 125
/* memory slots that are not exposed to userspace */
#define KVM_PRIVATE_MEM_SLOTS 3
#define KVM_MEM_SLOTS_NUM (KVM_USER_MEM_SLOTS + KVM_PRIVATE_MEM_SLOTS)

So it should be able to support many PCI-passthrough devices. To try
to take advantage of this, I have increased the number of KVM slots in
kvm-all.c:
75c75
<     KVMSlot slots[32];
---
>     KVMSlot slots[96];

I have also change the max number of vhost memory regions allowed by
the kernel from 64 to 128 to support more vhost devices
(VHOST_MEMORY_MAX_NREGIONS in drivers/vhost/vhost.c).

With these changes, I am able to pass 31 VFs to the guest without
problem, bringing the total number of PCI devices in the guest to 38:

00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.2 USB controller: Intel Corporation 82371SB PIIX3 USB
[Natoma/Triton II] (rev 01)
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:04.0 Unclassified device [00ff]: Red Hat, Inc Virtio memory balloon
00:05.0 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:05.1 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:05.2 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:05.3 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:05.4 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:05.5 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:05.6 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:05.7 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:06.0 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:06.1 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:06.2 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:06.3 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:06.4 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:06.5 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:06.6 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:06.7 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:07.0 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:07.1 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:07.2 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:07.3 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:07.4 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:07.5 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:07.6 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:07.7 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:08.0 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:08.1 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:08.2 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:08.3 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:08.4 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:08.5 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)
00:08.6 Ethernet controller: Intel Corporation 82599 Ethernet
Controller Virtual Function (rev 01)


However, when I try to add VF 32, I get the following error:
qemu-system-x86_64: /home/antoninb/qemu-1.6.0/exec.c:773:
phys_section_add: Assertion `next_map.sections_nb < (1 << 12)' failed.

Any thoughts on whether this could be easily patched?

Thanks,

Antonin