[Qemu-devel] Problems with Pass through of a AMD Vega 56/64

[James Courtier-Dutton]

Hi,

Problem:
Pass through of a AMD Vega 56 or 64 on a AMD Threadripper 1950X with X399
Motherboard , Host OS: Linux Kernel 5.1.5, did not work.

Various problems were observed:
1) PCI BAR resizing -  The GPU tries to resize the BAR from 256M to 8G. The
resize fails, but the Guest does not know this, and continues on as if it
was resized, resulting in a VM_L2_PROTECTION_FAULT_STATUS in the guest.
Symptoms of (1):
Able to passthru most PCI devices except the GPU.
Solution:
This problem has been worked-around with (See other email by Alex W):
[PATCH] vfio/pci: Hide Resizable BAR capability
The reason the GPU was problematic, is because it was the only device that
tried to resize the BAR.
Fortunately, the AMD GPU driver can work with a 256M or 8G BAR.
The real fix will be the implementation of support for PCI BAR Resizing in
QEMU and VFIO.

2) The QEMU fails to start with GPU passthrough, in that it does not reach
the Seabios messages and sits at 100% CPU.
I verified that the AMD GPU card's option ROM contained both Legacy BIOS
and EFI images in it.
Removing the GPU PCI device in virt-manager allows the VM to boot, but
adding the GPU PCI device causes it to fail to boot.
Symptoms:
VM won't start, left with blank screen.
Solution:
Using Q35 and OVMF EFI works.
Using 440 and Seabios does not work.
My guess is there is a bug in Seabios preventing it from working.

3) Use case:  Have 2 AMD Vega CPUs and wish only one to be passthru to
guest and the other to be used by the Host.
Problem: Currently you can passthru both, or neither.
Solution:
I have written a patch for VFIO to handle this, but it is still work in
progress before it will be accepted into kernel mainstream.

Kind Regards

James