Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists

[Gleb Natapov]

On Mon, Feb 20, 2012 at 08:15:15PM +0100, Peter Lieven wrote:
> On 20.02.2012 19:40, Gleb Natapov wrote:
> >On Mon, Feb 20, 2012 at 07:17:55PM +0100, Peter Lieven wrote:
> >>Hi,
> >>
> >>I came a across an issue with a Windows 7 (32-bit) as well as with a
> >>Windows 2008 R2 (64-bit) guest.
> >>
> >>If I transfer a file from the VM via CIFS or FTP to a remote machine,
> >>i get very poor read performance (around 13MB/s). The VM peaks at 100%
> >>cpu and I see a lot of insn_emulations and all kinds of exists in kvm_stat
> >>
> >>efer_reload                    0         0
> >>exits                    2260976     79620
> >>fpu_reload                  6197        11
> >>halt_exits                114734      5011
> >>halt_wakeup               111195      4876
> >>host_state_reload        1499659     60962
> >>hypercalls                     0         0
> >>insn_emulation           1577325     58488
> >>insn_emulation_fail            0         0
> >>invlpg                         0         0
> >>io_exits                  943949     40249
> >Hmm, too many of those.
> >
> >>irq_exits                 108679      5434
> >>irq_injections            236545     10788
> >>irq_window                  7606       246
> >>largepages                   672         5
> >>mmio_exits                460020     16082
> >>mmu_cache_miss               119         0
> >>mmu_flooded                    0         0
> >>mmu_pde_zapped                 0         0
> >>mmu_pte_updated                0         0
> >>mmu_pte_write              13474         9
> >>mmu_recycled                   0         0
> >>mmu_shadow_zapped            141         0
> >>mmu_unsync                     0         0
> >>nmi_injections                 0         0
> >>nmi_window                     0         0
> >>pf_fixed                   22803        35
> >>pf_guest                       0         0
> >>remote_tlb_flush             239         2
> >>request_irq                    0         0
> >>signal_exits                   0         0
> >>tlb_flush                  20933         0
> >>
> >>If I run the same VM with a Ubuntu 10.04.4 guest I get around 60MB/s
> >>throughput. The kvm_stats look a lot more sane.
> >>
> >>efer_reload                    0         0
> >>exits                    6132004     17931
> >>fpu_reload                 19863         3
> >>halt_exits                264961      3083
> >>halt_wakeup               236468      2959
> >>host_state_reload        1104468      3104
> >>hypercalls                     0         0
> >>insn_emulation           1417443      7518
> >>insn_emulation_fail            0         0
> >>invlpg                         0         0
> >>io_exits                  869380      2795
> >>irq_exits                 253501      2362
> >>irq_injections            616967      6804
> >>irq_window                201186      2161
> >>largepages                  1019         0
> >>mmio_exits                205268         0
> >>mmu_cache_miss               192         0
> >>mmu_flooded                    0         0
> >>mmu_pde_zapped                 0         0
> >>mmu_pte_updated                0         0
> >>mmu_pte_write            7440546         0
> >>mmu_recycled                   0         0
> >>mmu_shadow_zapped            259         0
> >>mmu_unsync                     0         0
> >>nmi_injections                 0         0
> >>nmi_window                     0         0
> >>pf_fixed                   38529        30
> >>pf_guest                       0         0
> >>remote_tlb_flush             761         1
> >>request_irq                    0         0
> >>signal_exits                   0         0
> >>tlb_flush                      0         0
> >>
> >>I use virtio-net (with vhost-net) and virtio-blk. I tried disabling
> >>hpet (which basically illiminated the mmio_exits, but does not
> >>increase
> >>performance) and also commit (39a7a362e16bb27e98738d63f24d1ab5811e26a8
> >>) - no improvement.
> >>
> >>My commandline:
> >>/usr/bin/qemu-kvm-1.0 -netdev
> >>type=tap,id=guest8,script=no,downscript=no,ifname=tap0,vhost=on
> >>-device virtio-net-pci,netdev=guest8,mac=52:54:00:ff:00:d3 -drive 
> >>format=host_device,file=/dev/mapper/iqn.2001-05.com.equallogic:0-8a0906-eeef4e007-a8a9f3818674f2fc-lieven-windows7-vc-r80788,if=virtio,cache=none,aio=native
> >>-m 2048 -smp 2 -monitor tcp:0:4001,server,nowait -vnc :1 -name
> >>lieven-win7-vc -boot order=dc,menu=off -k de -pidfile
> >>/var/run/qemu/vm-187.pid -mem-path /hugepages -mem-prealloc -cpu
> >>host -rtc base=localtime -vga std -usb -usbdevice tablet -no-hpet
> >>
> >>What further information is needed to debug this further?
> >>
> >Which kernel version (looks like something recent)?
> 2.6.38 with kvm-kmod 3.2
> >Which host CPU (looks like something old)?
> why? i guess its (quite) new.
> 
Yes, it is. I misread the kvm_stat data and thought that CPU does not
have ept, but it does.

> vendor_id    : GenuineIntel
> cpu family    : 6
> model        : 44
> model name    : Intel(R) Xeon(R) CPU           L5640  @ 2.27GHz
> stepping    : 2
> cpu MHz        : 1596.000
> cache size    : 12288 KB
> physical id    : 1
> siblings    : 6
> core id        : 10
> cpu cores    : 6
> apicid        : 52
> initial apicid    : 52
> fpu        : yes
> fpu_exception    : yes
> cpuid level    : 11
> wp        : yes
> flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
> syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts
> rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64
> monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1
> sse4_2 popcnt aes lahf_lm arat dts tpr_shadow vnmi flexpriority ept
> vpid
> bogomips    : 2254.43
> clflush size    : 64
> cache_alignment    : 64
> address sizes    : 40 bits physical, 48 bits virtual
> 
> >Which Windows' virtio drivers are you using?
> i used to use 0.1-16 and today also tried 0.1-22 from
> http://alt.fedoraproject.org/pub/alt/virtio-win/latest/images/
> >Take a trace like described here http://www.linux-kvm.org/page/Tracing
> >(with -no-hpet please).
> will prepare this.
> >Try to use -cpu host,+x2apic. It may help Linux guest performance.
> Thanks, it improved throughput a little while lowering the
> cpu usage. Windows does not support this?
> 
Not yet. It will support something similar when qemu will have hyper-v
support.

--
                        Gleb.