[Qemu-devel] Host kernel crash at pci_find_upstream_pcie_bridge on VM exit

[Ganesh Narayanaswamy]

Hi,

I am running qemu with kvm and VT-d enabled and a couple of PCI devices assigned to the guest VM. Both host and guest are running linux 2.6 kernel.  

The passthrough works fine, but when I exit the VM, the host kernel crashes with the following backtrace:

<4>[ 5569.836893] Process qemu-system-x86 (pid: 2925, threadinfo ffff8801f5f40000, task ffff88024fa28720)

<0>[ 5569.944946] Stack:

<4>[ 5569.968845]  ffff8801f5f41aa8 ffffffff811a45fb ffff88024f04b680 ffff88024f049980

<4>[ 5570.057156]  ffff88024f04b680 ffff88024f049988 ffff8801f5f41b08 ffffffff811a6371

<4>[ 5570.145470]  ffff8801f5f41ad8 ffffffff81391045 0000000000000246 ffff88024f049990

<0>[ 5570.233785] Call Trace:

<4>[ 5570.262880]  [<ffffffff811a45fb>] iommu_detach_dependent_devices+0x25/0x91

<4>[ 5570.344958]  [<ffffffff811a6371>] vm_domain_exit+0xf8/0x28b

<4>[ 5570.411457]  [<ffffffff81391045>] ? sub_preempt_count+0x92/0xa6

<4>[ 5570.482106]  [<ffffffff811a651a>] intel_iommu_domain_destroy+0x16/0x18

<4>[ 5570.560030]  [<ffffffff811fb5ea>] iommu_domain_free+0x16/0x22

<4>[ 5570.628611]  [<ffffffffa0006261>] kvm_iommu_unmap_guest+0x22/0x28 [kvm]

<4>[ 5570.707570]  [<ffffffffa0009b7b>] kvm_arch_destroy_vm+0x19/0x12a [kvm]

<4>[ 5570.785492]  [<ffffffffa0002614>] kvm_put_kvm+0xe6/0x129 [kvm]

<4>[ 5570.855102]  [<ffffffffa0002eb3>] kvm_vcpu_release+0x13/0x17 [kvm]

<4>[ 5570.928867]  [<ffffffff8109cdfc>] fput+0x117/0x1be

<4>[ 5570.986013]  [<ffffffff8109a147>] filp_close+0x63/0x6d

<4>[ 5571.047314]  [<ffffffff810342dd>] put_files_struct+0x6f/0xda

<4>[ 5571.114845]  [<ffffffff8103438e>] exit_files+0x46/0x4e

<4>[ 5571.176145]  [<ffffffff81035b3d>] do_exit+0x1fc/0x681

<4>[ 5571.236416]  [<ffffffffa000dedc>] ? kvm_arch_vcpu_ioctl_run+0xc2d/0xc55 [kvm]

<4>[ 5571.321605]  [<ffffffff8138cc41>] ? __mutex_lock_slowpath+0x26c/0x294

<4>[ 5571.398490]  [<ffffffff81036034>] do_group_exit+0x72/0x9a

<4>[ 5571.462907]  [<ffffffff8103fec9>] get_signal_to_deliver+0x331/0x350

<4>[ 5571.537719]  [<ffffffff81001f0f>] do_signal+0x6d/0x69a

<4>[ 5571.599013]  [<ffffffff811da1fc>] ? put_ldisc+0x92/0x97

<4>[ 5571.661353]  [<ffffffff810a95ea>] ? do_vfs_ioctl+0x527/0x576

<4>[ 5571.728887]  [<ffffffff81002563>] do_notify_resume+0x27/0x51

<4>[ 5571.796419]  [<ffffffff810a968c>] ? sys_ioctl+0x53/0x65

<4>[ 5571.858758]  [<ffffffff81002b9b>] int_signal+0x12/0x17

<0>[ 5571.920058] Code: 48 85 d2 0f 95 c0 c9 c3 55 80 7f 4a 00 48 89 f8 48 89 e5 75 46 31 d2 48 8b 40 10 48 83 78 10 00 75 05 48 89 d0 eb 36 48 8b 40 38 <80> 78 4a 00 48 89 c2 74 e3 80 78 4b 07 74 23 80 3d 86 b5 5a 00 

<1>[ 5572.145516] RIP  [<ffffffff81197f8c>] pci_find_upstream_pcie_bridge+0x23/0x57

<4>[ 5572.230712]  RSP <ffff8801f5f41a78>

Here is my qemu command line:

qemu-system-x86_64 -M q35 --enable-kvm -m 2048 -nographic -vga std -usb -drive file=<IMG file>,if=none,id=drive-sata-disk0,format=raw -device ahci,id=ahci -device ide-drive,bus=ahci.0,drive=drive-sata-disk0,id=sata-disk0,bootindex=1 -device pci-assign,host=04:00.0 -device pci-assign,host=03:00.0

The two PCI devices in question are behind a PCIe bridge which is connected to the rootport. The crash seems to be happening when  cleaning up the PCIe tree of the passed-through PCI devices.  I tried passing through the downstream ports of the bridge as well, but that is not supported by qemu.

Am I doing something wrong/unexpected here ? Any help in understanding this issue will help me fix the issue properly.

Thanks,

Ganesh