[Qemu-devel] Qemu deadlocks in tb_lock when using SVM+SoftMMU

[Alexander Boettcher]

Hello,

beginning with commit 3bd1d74576bacb120949e13cdeded7a0c792c685

"cputlb: introduce tlb_flush_* async work"

using Qemu with SoftMMU+SVM virtualization deadlocks because tb_lock is
taken second time in cputlb.c tlb_flush_nocheck() function. The first
time tb_lock is taken, according to my debugging, in cpu-exex.c
tb_find() line 361.

I'm using Qemu with:

qemu-system-x86_64 -s -no-kvm -display sdl -m 512 -cpu phenom -nographic
-cdrom genode.iso

When building with
./configure --target-list=x86_64-softmmu --enable-debug --disable-pie
--enable-debug-tcg

I get also a

translate-all.c:165: tb_lock: Assertion `!have_tb_lock' failed.

beginning with commit 3bd1d74576bacb120949e13cdeded7a0c792c685. Before
the commit all is fine.

Since I'm not very familiar with Qemu internals, it is not clear to me
whether this commit breaks things or whether something must be
handled/added special somewhere else. I attached below the backtrace of
Qemu when it hangs in tb_lock.

In [0] my branch based on 3bd1d74576bacb120949e13cdeded7a0c792c685 is
used and [1] contains the iso image, if somebody wants try to reproduce it.

[0] https://github.com/alex-ab/qemu/commits/genode_svm_issue
[1]
https://github.com/alex-ab/qemu/commit/1130fee3b04dd2bee576241de9a5771d6855b327

Thanks in advance,

Alex.

-- 
Alexander Boettcher
Genode Labs

http://www.genode-labs.com - http://www.genode.org

Genode Labs GmbH - Amtsgericht Dresden - HRB 28424 - Sitz Dresden
Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth




[init -> log_terminal]
[init -> log_terminal] [ 0] CORE:0:0:0 10:2:3:0 [0] AMD Phenom(tm) 9550
Quad-Core Processor
qemu-system-x86_64: qemu.git/translate-all.c:165: tb_lock: Assertion
`!have_tb_lock' failed.




(gdb) info threads
  Id   Target Id         Frame
* 1    Thread 0x7ffbc19d3c00 (LWP 8396) "qemu-system-x86"
0x00007ffbbfd2ac21 in __GI_ppoll (fds=0x273a330, nfds=6,
    timeout=<optimized out>, sigmask=0x0) at
../sysdeps/unix/sysv/linux/ppoll.c:50
  2    Thread 0x7ffbbb970700 (LWP 8397) "qemu-system-x86" syscall () at
../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
  3    Thread 0x7ffbb8206700 (LWP 8399) "qemu-system-x86" __lll_lock_wait ()
    at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
(gdb) thread 3
[Switching to thread 3 (Thread 0x7ffbb8206700 (LWP 8399))]
#0  __lll_lock_wait () at
../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
135     ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S: No such file or
directory.
(gdb) bt
#0  __lll_lock_wait () at
../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1  0x00007ffbc0002dbd in __GI___pthread_mutex_lock (mutex=0xf6ea18
<tcg_ctx+280>) at ../nptl/pthread_mutex_lock.c:80
#2  0x000000000089852c in qemu_mutex_lock (mutex=0xf6ea18 <tcg_ctx+280>)
at util/qemu-thread-posix.c:60
#3  0x0000000000416103 in tb_lock () at qemu.git/translate-all.c:166
#4  0x000000000046e8d7 in tlb_flush_nocheck (cpu=0x164a360) at
qemu.git/cputlb.c:93
#5  0x000000000046ea2e in tlb_flush (cpu=0x164a360) at qemu.git/cputlb.c:121
#6  0x0000000000538987 in cpu_x86_update_cr4 (env=0x16525f0, new_cr4=1784)
    at qemu.git/target/i386/helper.c:660
#7  0x000000000055e318 in cpu_vmexit (env=0x16525f0, exit_code=78,
exit_info_1=4, retaddr=0)
    at qemu.git/target/i386/svm_helper.c:689
#8  0x000000000055d9b7 in cpu_svm_check_intercept_param (env=0x16525f0,
type=78, param=4, retaddr=0)
    at qemu.git/target/i386/svm_helper.c:511
#9  0x0000000000541acf in raise_interrupt2 (env=0x16525f0, intno=14,
is_int=0, error_code=4, next_eip_addend=0, retaddr=0)
    at qemu.git/target/i386/excp_helper.c:96
#10 0x0000000000541c0d in raise_exception_err_ra (env=0x16525f0,
exception_index=14, error_code=4, retaddr=0)
    at qemu.git/target/i386/excp_helper.c:127
#11 0x00000000005621a9 in tlb_fill (cs=0x164a360, addr=1245184,
access_type=MMU_INST_FETCH, mmu_idx=1, retaddr=0)
    at qemu.git/target/i386/mem_helper.c:212
#12 0x0000000000476c15 in helper_ret_ldb_cmmu (env=0x16525f0,
addr=1245184, oi=1, retaddr=0)
    at qemu.git/softmmu_template.h:127
#13 0x000000000051c86e in cpu_ldub_code_ra (env=0x16525f0, ptr=1245184,
retaddr=0)
    at qemu.git/include/exec/cpu_ldst_template.h:102
#14 0x000000000051c8e4 in cpu_ldub_code (env=0x16525f0, ptr=1245184)
    at qemu.git/include/exec/cpu_ldst_template.h:114
#15 0x0000000000522182 in insn_get (env=0x16525f0, s=0x7ffbb82057e0,
ot=MO_8)
    at qemu.git/target/i386/translate.c:2107
#16 0x000000000052ff3c in disas_insn (env=0x16525f0, s=0x7ffbb82057e0,
pc_start=1245183)
    at qemu.git/target/i386/translate.c:6520
#17 0x0000000000536458 in gen_intermediate_code (env=0x16525f0,
tb=0x7ffbb9ce3a38)
    at qemu.git/target/i386/translate.c:8449
---Type <return> to continue, or q <return> to quit---
#18 0x0000000000417616 in tb_gen_code (cpu=0x164a360, pc=1245179,
cs_base=0, flags=7342771, cflags=0)
    at qemu.git/translate-all.c:1281
#19 0x000000000041993c in tb_find (cpu=0x164a360, last_tb=0x0,
tb_exit=0) at qemu.git/cpu-exec.c:370
#20 0x000000000041a25b in cpu_exec (cpu=0x164a360) at
qemu.git/cpu-exec.c:685
#21 0x000000000044b078 in tcg_cpu_exec (cpu=0x164a360) at
qemu.git/cpus.c:1251
#22 0x000000000044b2e7 in qemu_tcg_rr_cpu_thread_fn (arg=0x164a360) at
qemu.git/cpus.c:1347
#23 0x00007ffbc00006ba in start_thread (arg=0x7ffbb8206700) at
pthread_create.c:333
#24 0x00007ffbbfd3682d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:109