qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-devel] Multi GPU passthrough via VFIO


From: Maik Broemme
Subject: [Qemu-devel] Multi GPU passthrough via VFIO
Date: Wed, 5 Feb 2014 19:59:45 +0100
User-agent: Mutt/1.5.21 (2010-09-15)

Hi,

currently VFIO with multi GPU passthrough is working partially and
hopefully somebody has a hint about the problem. I'm doing passthrough
of an AMD Radeon R9 290X and AMD Radeon 7870 GHz Edition to a single VM.

If the VM is running Linux this works quite well with radeon or fglrx
driver. Please see 'dmesg' log attached, when using the radeon driver.
If needed I can also post one with fglrx driver.

If I do the exact same passthrough to a Windows VM and use latest AMD
Catalyst 14.1 (2/1/2014) or AMD Catalyst 13.12 (12/18/2013) I can get
only the first device working (AMD R9 290X) with 'x-vga=on'. I don't
enable 'x-vga=on' on second device as this should never work. :) I see
BIOS boot screen and everything works fine except for the second GPU.
The windows device manager just show me "Code 12" for the second GPU
and its HD Audio device. Code 12 means: "This device cannot find enough
free resources that it can use".

QEMU is called in both cases via the following. I just replace the
'-drive' accordingly.

/usr/bin/taskset -c 0,1,2,3 /usr/bin/qemu-system-x86_64 \
  -machine q35,accel=kvm \
  -enable-kvm \
  -nodefaults \
  -nographic \
  -vga none \
  -boot order=nc \
  -cpu host \
  -smp cores=4,threads=1,sockets=1 \
  -m 8192 \
  -rtc base=localtime \
  -k de \
  -drive 
file=/srv/kvm/linux-drive0.img,id=drive0,if=none,cache=none,aio=threads \
  -mon chardev=monitor0 \
  -chardev socket,id=monitor0,path=/tmp/linux.monitor,nowait,server \
  -netdev tap,id=net0,vhost=on,helper=/usr/lib/qemu/qemu-bridge-helper \
  -device virtio-net-pci,netdev=net0,mac=00:00:00:02:01:04 \
  -device virtio-blk-pci,drive=drive0,ioeventfd=on \
  -device ioh3420,bus=pcie.0,id=pcie0,port=1,chassis=1,multifunction=on \
  -device ioh3420,bus=pcie.0,id=pcie1,port=2,chassis=2,multifunction=on \
  -device vfio-pci,host=01:00.0,addr=00.0,bus=pcie0,multifunction=on,x-vga=on \
  -device vfio-pci,host=01:00.1,addr=00.1,bus=pcie0 \
  -device vfio-pci,host=02:00.0,addr=00.0,bus=pcie1,multifunction=on \
  -device vfio-pci,host=02:00.1,addr=00.1,bus=pcie1 \
  -no-reboot

My setup is the following:

Kernel: linux-3.13.1
Seabios: seabios-git-rel.1.7.4.r51.g151d034 (5/2/2014)
QEMU: qemu-git-2.0.r30666.g31db5b3 (5/2/2014)

Below is the 'lspci' output and I'm using the AMD Radeon HD 5430 as device
for my local X server:

00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI 
bridge (external gfx0 port B) (rev 02)
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory 
Management Unit (IOMMU)
00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI 
bridge (PCI express gpp port B)
00:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI 
bridge (PCI express gpp port D)
00:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI 
bridge (PCI express gpp port H)
00:0d.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI 
bridge (external gfx1 port B)
00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI] 
SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller 
(rev 42)
00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia 
(Intel HDA) (rev 40)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 
LPC host controller (rev 40)
00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI 
Bridge (rev 40)
00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
00:15.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 
PCI to PCI bridge (PCIE port 0)
00:15.1 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 
PCI to PCI bridge (PCIE port 1)
00:15.2 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI to PCI 
bridge (PCIE port 2)
00:15.3 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI to PCI 
bridge (PCIE port 3)
00:16.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:16.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor 
Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor 
Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor 
Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor 
Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor 
Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor 
Function 5
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] 
Hawaii XT [Radeon HD 8970]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Device aac8
02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] 
Pitcairn XT [Radeon HD 7870 GHz Edition]
02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape 
Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
03:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller 
(rev 01)
04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Park 
[Mobility Radeon HD 5430]
04:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cedar HDMI Audio 
[Radeon HD 5400/6300 Series]
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 
PCI Express Gigabit Ethernet Controller (rev 06)
07:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller 
(rev 01)

Another minor issue is that the R9 290X is not reset during shutdown of
VM (neither Linux nor Windows) but it can be tricked with doing
"suspend-to-ram" between two starts. That's why I use '-no-reboot' option
in QEMU. The 7870 is doing the reset properly.

--Maik

Attachment: dmesg.log
Description: Text document


reply via email to

[Prev in Thread] Current Thread [Next in Thread]