[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] [PATCH v4 07/12] tcg: Prepare TB invalidation for lockless
From: |
Sergey Fedorov |
Subject: |
[Qemu-devel] [PATCH v4 07/12] tcg: Prepare TB invalidation for lockless TB lookup |
Date: |
Fri, 15 Jul 2016 20:58:47 +0300 |
From: Sergey Fedorov <address@hidden>
When invalidating a translation block, set an invalid CPU state into the
TranslationBlock structure first.
As soon as the TB is marked with an invalid CPU state, there is no need
to remove it from CPU's 'tb_jmp_cache'. However it will be necessary to
recheck whether the target TB is still valid after acquiring 'tb_lock'
but before calling tb_add_jump() since TB lookup is to be performed out
of 'tb_lock' in future. Note that we don't have to check 'last_tb' since
it is safe to patch an already invalidated TB since it will not be
executed anyway.
Suggested-by: Paolo Bonzini <address@hidden>
Signed-off-by: Sergey Fedorov <address@hidden>
Signed-off-by: Sergey Fedorov <address@hidden>
Reviewed-by: Alex Bennée <address@hidden>
---
Changes in v4:
- smp_wmb() removed after tb_mark_invalid()
- atomic access to TB CPU state
---
cpu-exec.c | 7 ++++---
include/exec/exec-all.h | 8 +++++++-
translate-all.c | 11 ++---------
3 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/cpu-exec.c b/cpu-exec.c
index c973e3b85922..e16df762f50a 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -334,8 +334,9 @@ static inline TranslationBlock *tb_find_fast(CPUState *cpu,
cpu_get_tb_cpu_state(env, &pc, &cs_base, &flags);
tb_lock();
tb = atomic_read(&cpu->tb_jmp_cache[tb_jmp_cache_hash_func(pc)]);
- if (unlikely(!tb || tb->pc != pc || tb->cs_base != cs_base ||
- tb->flags != flags)) {
+ if (unlikely(!tb || atomic_read(&tb->pc) != pc ||
+ atomic_read(&tb->cs_base) != cs_base ||
+ atomic_read(&b->flags) != flags)) {
tb = tb_find_slow(cpu, pc, cs_base, flags);
}
#ifndef CONFIG_USER_ONLY
@@ -352,7 +353,7 @@ static inline TranslationBlock *tb_find_fast(CPUState *cpu,
/* Check if translation buffer has been flushed */
if (cpu->tb_flushed) {
cpu->tb_flushed = false;
- } else {
+ } else if (!tb_is_invalid(tb)) {
tb_add_jump(last_tb, tb_exit, tb);
}
}
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index a499c7c56eef..8f0afcdbd62a 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -258,7 +258,13 @@ void tb_phys_invalidate(TranslationBlock *tb,
tb_page_addr_t page_addr);
static inline void tb_mark_invalid(TranslationBlock *tb)
{
- cpu_get_invalid_tb_cpu_state(&tb->pc, &tb->cs_base, &tb->flags);
+ target_ulong pc = 0, cs_base = 0;
+ uint32_t flags = 0;
+
+ cpu_get_invalid_tb_cpu_state(&pc, &cs_base, &flags);
+ atomic_set(&tb->pc, pc);
+ atomic_set(&tb->cs_base, cs_base);
+ atomic_set(&tb->flags, flags);
}
static inline bool tb_is_invalid(TranslationBlock *tb)
diff --git a/translate-all.c b/translate-all.c
index 788fed1e0765..9db72e8982b1 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -986,11 +986,12 @@ static inline void tb_jmp_unlink(TranslationBlock *tb)
/* invalidate one TB */
void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr)
{
- CPUState *cpu;
PageDesc *p;
uint32_t h;
tb_page_addr_t phys_pc;
+ tb_mark_invalid(tb);
+
/* remove the TB from the hash list */
phys_pc = tb->page_addr[0] + (tb->pc & ~TARGET_PAGE_MASK);
h = tb_hash_func(phys_pc, tb->pc, tb->flags);
@@ -1008,14 +1009,6 @@ void tb_phys_invalidate(TranslationBlock *tb,
tb_page_addr_t page_addr)
invalidate_page_bitmap(p);
}
- /* remove the TB from the hash list */
- h = tb_jmp_cache_hash_func(tb->pc);
- CPU_FOREACH(cpu) {
- if (atomic_read(&cpu->tb_jmp_cache[h]) == tb) {
- atomic_set(&cpu->tb_jmp_cache[h], NULL);
- }
- }
-
/* suppress this TB from the two jump lists */
tb_remove_from_jmp_list(tb, 0);
tb_remove_from_jmp_list(tb, 1);
--
2.9.1
- [Qemu-devel] [PATCH v4 00/12] Reduce lock contention on TCG hot-path, Sergey Fedorov, 2016/07/15
- [Qemu-devel] [PATCH v4 02/12] tcg: Pass last_tb by value to tb_find_fast(), Sergey Fedorov, 2016/07/15
- [Qemu-devel] [PATCH v4 03/12] tcg: Prepare safe tb_jmp_cache lookup out of tb_lock, Sergey Fedorov, 2016/07/15
- [Qemu-devel] [PATCH v4 01/12] util/qht: Document memory ordering assumptions, Sergey Fedorov, 2016/07/15
- [Qemu-devel] [PATCH v4 04/12] tcg: Prepare safe access to tb_flushed out of tb_lock, Sergey Fedorov, 2016/07/15
- [Qemu-devel] [PATCH v4 08/12] tcg: set up tb->page_addr before insertion, Sergey Fedorov, 2016/07/15
- [Qemu-devel] [PATCH v4 07/12] tcg: Prepare TB invalidation for lockless TB lookup,
Sergey Fedorov <=
- [Qemu-devel] [PATCH v4 05/12] target-i386: Remove redundant HF_SOFTMMU_MASK, Sergey Fedorov, 2016/07/15
- [Qemu-devel] [PATCH v4 06/12] tcg: Introduce tb_mark_invalid() and tb_is_invalid(), Sergey Fedorov, 2016/07/15
- [Qemu-devel] [PATCH v4 09/12] tcg: cpu-exec: remove tb_lock from the hot-path, Sergey Fedorov, 2016/07/15
- [Qemu-devel] [PATCH v4 10/12] tcg: Avoid bouncing tb_lock between tb_gen_code() and tb_add_jump(), Sergey Fedorov, 2016/07/15
- [Qemu-devel] [PATCH v4 12/12] tcg: rename tb_find_physical(), Sergey Fedorov, 2016/07/15
- [Qemu-devel] [PATCH v4 11/12] tcg: Merge tb_find_slow() and tb_find_fast(), Sergey Fedorov, 2016/07/15
- Re: [Qemu-devel] [PATCH v4 00/12] Reduce lock contention on TCG hot-path, Paolo Bonzini, 2016/07/16