More than 255 vcpus Windows VM setup without viommu ?

[Sandesh Patel]

Hi All,
Is it possible to setup a large Windows VM (say 512 vcpus) without adding 
viommu (EIM=on, IR=on)?
When I try to power such VM, the qemu process crashes with error-
```
qemu-kvm: ../accel/kvm/kvm-all.c:1837: kvm_irqchip_commit_routes: Assertion 
`ret == 0’ failed

```
Stack trace-

```
#1  0x00007f484bc21ea5 abort (libc.so.6)
#2  0x00007f484bc21d79 __assert_fail_base.cold.0 (libc.so.6)
#3  0x00007f484bc47426 __assert_fail (libc.so.6)
#4  0x000055b7215634d3 kvm_irqchip_commit_routes (qemu-kvm)
#5  0x000055b7213bfc7e kvm_virtio_pci_vector_use_one (qemu-kvm)
#6  0x000055b7213c02cf virtio_pci_set_guest_notifiers (qemu-kvm)
#7  0x000055b7214dd848 vhost_scsi_common_start (qemu-kvm)
#8  0x000055b72139b936 vhost_user_scsi_start (qemu-kvm)
#9  0x000055b72139ba64 vhost_user_scsi_set_status (qemu-kvm)
#10 0x000055b7214f865a virtio_set_status (qemu-kvm)
#11 0x000055b7213bdc3f virtio_pci_common_write (qemu-kvm)
#12 0x000055b721514e68 memory_region_write_accessor (qemu-kvm)
#13 0x000055b72151489e access_with_adjusted_size (qemu-kvm)
#14 0x000055b721514b89 memory_region_dispatch_write (qemu-kvm)
#15 0x000055b72151e3fc flatview_write_continue (qemu-kvm)
#16 0x000055b72151e553 flatview_write (qemu-kvm)
#17 0x000055b72151ee76 address_space_write (qemu-kvm)
#18 0x000055b721565526 kvm_cpu_exec (qemu-kvm)
#19 0x000055b72156634d kvm_vcpu_thread_fn (qemu-kvm)
#20 0x000055b721750224 qemu_thread_start (qemu-kvm)
#21 0x00007f484c0081ca start_thread (libpthread.so.0)
#22 0x00007f484bc39e73 
```

The error is due to invalid MSIX routing entry passed to KVM.

The VM boots fine if we attach a vIOMMU but adding a vIOMMU can potentially 
result in IO performance loss in guest.
I was interested to know if someone could boot a large Windows VM by some other 
means like kvm-msi-ext-dest-id.

Overheads of viommu have been shown for example in - 
https://static.sched.com/hosted_files/kvmforum2021/da/vIOMMU%20KVM%20Forum%202021%20-%20v4.pdf

Thanks and regards,
Sandesh