[Qemu-devel] PCI: Hot-removing a virtio-blk causes guest panic

[Amos Kong]

good: 3.3.0 guest kernel & qemu-kvm-rhel6
guest panic:  3.3.0 guest kernel & qemu-upstream (contains fix [1])

I didn't change anything of guest kernel,
It seems a bug of qemu-upstream.

[1] http://marc.info/?l=qemu-devel&m=133670266801022&w=2
    [PATCH] qom: fix refcounting in object_property_del_child()


>>> Start VM with one block device:
qemu-upstream --enable-kvm  -name 'vm1' -nodefaults -drive
file='nolvm.qcow2',index=0,if=virtio,cache=none,snapshot=on -net none -m
2000 -smp 2 -vnc :0  -kernel vmlinuz-3.3.0 -append 'ro root=/dev/vda1
console=tty0 console=ttyS0,115200'   -drive
file=images/u0,if=none,id=drive-virtio0-0-0,format=qcow2,cache=none
-device virtio-blk-pci,drive=drive-virtio0-0-0,id=virti0-0-0 -monitor
unix:/tmp/m,nowait,server

>>> hot-remove the virtio disk
(qemu)# echo "device_del virti0-0-0" | nc -U /tmp/m

>>> guest panic:
kernel BUG at drivers/virtio/virtio.c:158!
invalid opcode: 0000 [#1] SMP
CPU 0
Modules linked in:

Pid: 39, comm: kworker/0:2 Not tainted 3.3.0pcifix+ #46 Bochs Bochs
RIP: 0010:[<ffffffff8133da29>]  [<ffffffff8133da29>]
virtio_dev_remove+0x49/0x50
RSP: 0000:ffff880078a6da80  EFLAGS: 00010286
RAX: 00000000000000ff RBX: ffff8800788da800 RCX: 0000000000000000
RDX: 000000000000c052 RSI: 0000000000000202 RDI: 000000000001c052
RBP: ffff880078a6da90 R08: 0000000000000000 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000001 R12: ffffffff81ab7120
R13: ffffffff81a8e120 R14: ffff88007914f000 R15: ffff88007914f000
FS:  0000000000000000(0000) GS:ffff88007cc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fe852c9e008 CR3: 0000000077b33000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process kworker/0:2 (pid: 39, threadinfo ffff880078a6c000, task
ffff880078a6b4e0)
Stack:
 ffff88007914f000 ffff8800788da808 ffff880078a6dab0 ffffffff813949fc
 ffff8800788da868 ffff8800788da808 ffff880078a6dad0 ffffffff81394b4d
 ffff8800788da808 0000000000000005 ffff880078a6db00 ffffffff81393acf
Call Trace:
 [<ffffffff813949fc>] __device_release_driver+0x7c/0xe0
 [<ffffffff81394b4d>] device_release_driver+0x2d/0x40
 [<ffffffff81393acf>] bus_remove_device+0x10f/0x180
 [<ffffffff813918d0>] device_del+0x120/0x1d0
 [<ffffffff813919a2>] device_unregister+0x22/0x60
 [<ffffffff8133db92>] unregister_virtio_device+0x12/0x20
 [<ffffffff815653e9>] virtio_pci_remove+0x2a/0x6c
 [<ffffffff812ca8e2>] pci_device_remove+0x52/0x120
 [<ffffffff813949fc>] __device_release_driver+0x7c/0xe0
 [<ffffffff81394b4d>] device_release_driver+0x2d/0x40
 [<ffffffff81393acf>] bus_remove_device+0x10f/0x180
 [<ffffffff813918d0>] device_del+0x120/0x1d0
 [<ffffffff813919a2>] device_unregister+0x22/0x60
 [<ffffffff812c4664>] pci_stop_bus_device+0x94/0xa0
 [<ffffffff812dbe9c>] disable_device+0xac/0x190
 [<ffffffff81069694>] ? insert_work+0x34/0x80
 [<ffffffff812dc170>] acpiphp_disable_slot+0x30/0x60
 [<ffffffff812dc695>] acpiphp_check_bridge+0x35/0xf0
 [<ffffffff812dc941>] _handle_hotplug_event_func+0x121/0x1d0
 [<ffffffff81301e8c>] ? acpi_os_wait_events_complete+0x23/0x23
 [<ffffffff812dc820>] ? check_sub_bridges+0xd0/0xd0
 [<ffffffff8106ac62>] process_one_work+0x132/0x450
 [<ffffffff8106ca6b>] worker_thread+0x17b/0x3c0
 [<ffffffff8106c8f0>] ? manage_workers+0x120/0x120
 [<ffffffff8107196e>] kthread+0x9e/0xb0
 [<ffffffff8157f2a4>] kernel_thread_helper+0x4/0x10
 [<ffffffff810718d0>] ? kthread_freezable_should_stop+0x70/0x70
 [<ffffffff8157f2a0>] ? gs_change+0x13/0x13
Code: 90 00 00 00 48 8b 83 a8 02 00 00 48 89 df ff 50 10 84 c0 75 16 48
89 df be 01 00 00 00 e8 90 fd ff ff 48 83 c4 08 31 c0 5b c9 c3 <0f> 0b
eb fe 0f 1f 00 55 48 89 e5 41 55 41 54 53 48 83 ec 08 66
RIP  [<ffffffff8133da29>] virtio_dev_remove+0x49/0x50
 RSP <ffff880078a6da80>
---[ end trace aafd6463605a97fc ]---
BUG: unable to handle kernel paging request at fffffffffffffff8
IP: [<ffffffff81071430>] kthread_data+0x10/0x20
PGD 1a0d067 PUD 1a0e067 PMD 0
Oops: 0000 [#2] SMP
CPU 0
Modules linked in:

Pid: 39, comm: kworker/0:2 Tainted: G      D      3.3.0pcifix+ #46 Bochs
Bochs
RIP: 0010:[<ffffffff81071430>]  [<ffffffff81071430>] kthread_data+0x10/0x20
RSP: 0000:ffff880078a6d768  EFLAGS: 00010096
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: ffffffff81d563c0 RSI: 0000000000000000 RDI: ffff880078a6b4e0
RBP: ffff880078a6d768 R08: ffff880078a6b550 R09: 0000000000000000
R10: 0000000000000002 R11: 0000000000000000 R12: 0000000000000000
R13: ffff880078a6ba88 R14: 0000000000000001 R15: 0000000000000006
FS:  0000000000000000(0000) GS:ffff88007cc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: fffffffffffffff8 CR3: 0000000077b33000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process kworker/0:2 (pid: 39, threadinfo ffff880078a6c000, task
ffff880078a6b4e0)
Stack:
 ffff880078a6d788 ffffffff8106a1b5 ffff880078a6d788 ffff88007cc13500
 ffff880078a6d818 ffffffff81574ae3 ffff880078a6dfd8 0000000000013500
 ffff880078a6c010 0000000000013500 0000000000013500 0000000000013500
Call Trace:
 [<ffffffff8106a1b5>] wq_worker_sleeping+0x15/0xa0
 [<ffffffff81574ae3>] __schedule+0x5a3/0x730
 [<ffffffff81574fa9>] schedule+0x29/0x70
 [<ffffffff81054bbd>] do_exit+0x2ad/0x450
 [<ffffffff81576cac>] oops_end+0xac/0xf0
 [<ffffffff810167fb>] die+0x5b/0x90
 [<ffffffff81576804>] do_trap+0xc4/0x170
 [<ffffffff810146a5>] do_invalid_op+0x95/0xb0
 [<ffffffff8133da29>] ? virtio_dev_remove+0x49/0x50
 [<ffffffff812a27bc>] ? kobject_cleanup+0x9c/0x1b0
 [<ffffffff812a28dd>] ? kobject_release+0xd/0x10
 [<ffffffff812a262c>] ? kobject_put+0x2c/0x60
 [<ffffffff8157f11b>] invalid_op+0x1b/0x20
 [<ffffffff8133da29>] ? virtio_dev_remove+0x49/0x50
 [<ffffffff813949fc>] __device_release_driver+0x7c/0xe0
 [<ffffffff81394b4d>] device_release_driver+0x2d/0x40
 [<ffffffff81393acf>] bus_remove_device+0x10f/0x180
 [<ffffffff813918d0>] device_del+0x120/0x1d0
 [<ffffffff813919a2>] device_unregister+0x22/0x60
 [<ffffffff8133db92>] unregister_virtio_device+0x12/0x20
 [<ffffffff815653e9>] virtio_pci_remove+0x2a/0x6c
 [<ffffffff812ca8e2>] pci_device_remove+0x52/0x120
 [<ffffffff813949fc>] __device_release_driver+0x7c/0xe0
 [<ffffffff81394b4d>] device_release_driver+0x2d/0x40
 [<ffffffff81393acf>] bus_remove_device+0x10f/0x180
 [<ffffffff813918d0>] device_del+0x120/0x1d0
 [<ffffffff813919a2>] device_unregister+0x22/0x60
 [<ffffffff812c4664>] pci_stop_bus_device+0x94/0xa0
 [<ffffffff812dbe9c>] disable_device+0xac/0x190
 [<ffffffff81069694>] ? insert_work+0x34/0x80
 [<ffffffff812dc170>] acpiphp_disable_slot+0x30/0x60
 [<ffffffff812dc695>] acpiphp_check_bridge+0x35/0xf0
 [<ffffffff812dc941>] _handle_hotplug_event_func+0x121/0x1d0
 [<ffffffff81301e8c>] ? acpi_os_wait_events_complete+0x23/0x23
 [<ffffffff812dc820>] ? check_sub_bridges+0xd0/0xd0
 [<ffffffff8106ac62>] process_one_work+0x132/0x450
 [<ffffffff8106ca6b>] worker_thread+0x17b/0x3c0
 [<ffffffff8106c8f0>] ? manage_workers+0x120/0x120
 [<ffffffff8107196e>] kthread+0x9e/0xb0
 [<ffffffff8157f2a4>] kernel_thread_helper+0x4/0x10
 [<ffffffff810718d0>] ? kthread_freezable_should_stop+0x70/0x70
 [<ffffffff8157f2a0>] ? gs_change+0x13/0x13
Code: 66 66 66 90 65 48 8b 04 25 80 c6 00 00 48 8b 80 50 05 00 00 8b 40
f0 c9 c3 66 90 55 48 89 e5 66 66 66 66 90 48 8b 87 50 05 00 00 <48> 8b
40 f8 c9 c3 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 66
RIP  [<ffffffff81071430>] kthread_data+0x10/0x20
 RSP <ffff880078a6d768>
CR2: fffffffffffffff8
---[ end trace aafd6463605a97fd ]---
Fixing recursive fault but reboot is needed!


-- 
                        Amos.