Re: [Qemu-devel] The maximum limit of virtual network device

[Laszlo Ersek]

On 07/06/17 11:24, Marcel Apfelbaum wrote:
> On 06/07/2017 11:31, Laszlo Ersek wrote:

>> > Now, I would normally recommend sticking with i440fx for simplicity.
>> However, each PCI bridge requires 4KB of IO space (meaning (1 + 5) * 4KB
>> = 24KB),  and OVMF on the i440fx does not support that much (only
>> 0x4000). So, I'll recommend Q35 for IO space purposes; OVMF on Q35
>> provides 0xA000 (40KB).
> 
> So if we use OVMF, going for Q35 gives us actually more IO space, nice!
> However recommending Q35 for IO space seems odd :)

OVMF used to have only 0x4000 bytes for PCI IO aperture, since the
beginning. In <https://bugzilla.redhat.com/show_bug.cgi?id=1333238> I
investigated how much I could grow the aperture. On Q35, it was possible
to grow it to 0xA000 bytes (but even then you have to disable vmport,
which sort of sits in the middle otherwise). On i440fx, the IO ports in
use by platform devices were so badly distributed that moving beyond
0x4000 was not possible. See in particular:

https://bugzilla.redhat.com/show_bug.cgi?id=1333238#c16
https://bugzilla.redhat.com/show_bug.cgi?id=1333238#c19

>> Therefore I guess the simplest example I can give now is:
>> - use Q35 (for a larger IO space),
>> - plug a DMI-PCI bridge into the root bridge,
>> - plug 5 PCI bridges into the DMI-PCI bridge,
>> - plug 31 NICs per PCI bridge, each NIC into a separate slot.
>>
> 
> The setup looks OK to me (assuming OVMF is needed, otherwise
> PC + pci-bridges will result in more devices),

OVMF is quite needed; Jiaxin is one of the edk2 networking maintainers,
and I think he's using QEMU and OVMF as a testbed for otherwise
physically-oriented UEFI development.

> I do have a little concern.
> We want to deprecate the dmi-pci bridge since it does not support
> hot-plug (for itself or devices behind it).
> Alexandr (CCed) is a GSOC student working on a generic
> pcie-pci bridge that can (eventually) be hot-plugged
> into a PCIe Root Port and keeps the machine cleaner.

Nice!

Please include an update to "docs/pcie.txt" in the scope :)

Thanks!
Laszlo