Dear All,
As a result of fixing the bug [1] I discovered that QEMU in pure emulation (TCG) sometimes misses page dirtying on the migration. This is happens at least in the version 2.0.0 and should, according to the code, be the same in the master as well.
The reason for that is that only pages missing from the TLB cache are fetched using the `tlb_fill` that calls `x86_cpu_handle_mmu_fault` and finally `stl_phys_notdirty` which marks the page as dirty in the ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION] bitmap.
However, if the page is in the TLB cache then no such code is run and page is never marked as dirty, making memory dump created by `savevm` different form the actual VM memory. Sometimes this leads to memory structures corruption during the migration that results in the VM kernel Oops. This is indeed happens very frequently with the data referenced by the APIC timer IRQ handler, because these structures are almost always in the TLB-cache especially on a non-busy VM. [2]
Sadly, just flushing the TLB on `ram_save_setup` will not be enough because the appropriate TLB entry must be flushed whenever the appropriate page is saved and marked clean. This however can not be done in a thread-safe manner due to the race between the translated and migration code.