[Qemu-devel] [Bug 1636217] Re: qemu-kvm 2.7 does not boot kvm VMs with virtio on top of VMware ESX

[Roman Kagan]

This is a KVM bug.  It has been fixed in mainstream Linux in

commit d391f1207067268261add0485f0f34503539c5b0
Author: Vitaly Kuznetsov <address@hidden>
Date:   Thu Jan 25 16:37:07 2018 +0100

    x86/kvm/vmx: do not use vm-exit instruction length for fast MMIO when 
running nested
    
    I was investigating an issue with seabios >= 1.10 which stopped working
    for nested KVM on Hyper-V. The problem appears to be in
    handle_ept_violation() function: when we do fast mmio we need to skip
    the instruction so we do kvm_skip_emulated_instruction(). This, however,
    depends on VM_EXIT_INSTRUCTION_LEN field being set correctly in VMCS.
    However, this is not the case.
    
    Intel's manual doesn't mandate VM_EXIT_INSTRUCTION_LEN to be set when
    EPT MISCONFIG occurs. While on real hardware it was observed to be set,
    some hypervisors follow the spec and don't set it; we end up advancing
    IP with some random value.
    
    I checked with Microsoft and they confirmed they don't fill
    VM_EXIT_INSTRUCTION_LEN on EPT MISCONFIG.
    
    Fix the issue by doing instruction skip through emulator when running
    nested.
    
    Fixes: 68c3b4d1676d870f0453c31d5a52e7e65c7448ae
    Suggested-by: Radim Krčmář <address@hidden>
    Suggested-by: Paolo Bonzini <address@hidden>
    Signed-off-by: Vitaly Kuznetsov <address@hidden>
    Acked-by: Michael S. Tsirkin <address@hidden>
    Signed-off-by: Radim Krčmář <address@hidden>


Although the commit mentions Hyper-V as L0 hypervisor, the same problem
pertains to ESXi.

The commit is included in v4.16.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1636217

Title:
  qemu-kvm 2.7 does not boot kvm VMs with virtio on top of VMware ESX

Status in QEMU:
  New

Bug description:
  After todays Proxmox update all my Linux VMs stopped booting.

  # How to reproduce
  - Have KVM on top of VMware ESX (I use VMware ESX 6)
  - Boot Linux VM with virtio Disk drive.

  
  # Result
  virtio based VMs do not boot anymore:

  address@hidden:/etc/pve/nodes/demotuxdc/qemu-server# grep virtio0 100.conf 
  bootdisk: virtio0
  virtio0: pvestorage:100/vm-100-disk-1.raw,discard=on,size=20G

  (initially with cache=writethrough, but that doesn´t matter)

  What happens instead is:

  - BIOS displays "Booting from harddisk..."
  - kvm process of VM loops at about 140% of Intel(R) Core(TM) i5-6260U CPU @ 
1.80GHz Skylake dual core CPU

  Disk of course has valid bootsector:

  address@hidden:/srv/pvestorage/images/100# file -sk vm-100-disk-1.raw 
  vm-100-disk-1.raw: DOS/MBR boot sector DOS/MBR boot sector DOS executable 
(COM), boot code
  address@hidden:/srv/pvestorage/images/100# head -c 2048 vm-100-disk-1.raw | 
hd | grep GRUB
  00000170  be 94 7d e8 2e 00 cd 18  eb fe 47 52 55 42 20 00  |..}.......GRUB .|

  
  # Workaround 1
  - Change disk from virtio0 to scsi0
  - Debian boots out of the box after this change
  - SLES 12 needs a rebuilt initrd
  - CentOS 7 too, but it seems that is not even enough and it still fails (even 
in hostonly="no" mode for dracut)

  
  # Workaround 2
  Downgrade pve-qemu-kvm 2.7.0-3 to 2.6.2-2.

  
  # Expected results
  Disk boots just fine via virtio like it did before.

  
  # Downstream bug report
  Downstream suggests an issue with upstream qemu-kvm:

  https://bugzilla.proxmox.com/show_bug.cgi?id=1181

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1636217/+subscriptions