[Qemu-discuss] Got a blue screen from vm with nested virtualization

[John Y.]

I created 5 vms inside vm with kvm's nested virtualization.

1 of them booted fail and showed the bluescreen.

2 of them booted successfully, ran for several minutes and showed the bluescreen.

I got different bluescreen:

1. IRQL_NOT_LESS_OR_EQUAL

*** STOP: 0x0000000A (0x0000000000009007, 0x0000000000000002, 0x0000000000000000, 0xFFFFF800016CDC63)

2. PAGE_FAULT_IN_NONPAGED_AREA

*** STOP: 0x00000050 (0xFFFFF880019BD110, 0x0000000000000001, 0xFFFFF800019E788D, 0x0000000000000000)

3. 

*** STOP: 0x0000007E (0xFFFFFFFFC0000005, 0xFFFFF800015A7F8C, 0xFFFFF88002848408 0xFFFFFF99002947C60)

libvirt of VM in physical host:

<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">  

  <name>nestedtest</name>  

  <description>GCloud</description>  

  <uuid>da94c283-cf75-40d4-b08c-c5a65db1ca13</uuid>  

  <os> 

    <type>hvm</type>  

    <boot dev="hd"/> 

  </os>  

  <features> 

    <acpi/> 

  </features>  

  <clock offset="localtime"/>  

  <on_poweroff>destroy</on_poweroff>  

  <on_reboot>restart</on_reboot>  

  <on_crash>destroy</on_crash>  

  <vcpu current="8">8</vcpu>  

  <cpu mode="host-passthrough"/>  

  <memory>16777216</memory>  

  <currentMemory>16777216</currentMemory>  

  <devices> 

   <emulator>/root/qemu/x86_64-softmmu/qemu-system-x86_64</emulator> 

   <disk device="disk" type="file"> 

      <driver name="qemu" type="qcow2" cache="none"/>  

      <source file="/home/nested_test_snap"/>

      <target bus="virtio" dev="vda"/> 

    </disk>  

    <disk type="file" device="cdrom"> 

      <source dev=""/>  

      <target dev="hdc" bus="ide"/> 

    </disk>  

    <serial type="pty"> 

      <source path="/dev/pts/1"/>  

      <target port="0"/> 

    </serial>  

    <controller type="usb" index="0" model="ich9-ehci1"/>  

    <controller type="usb" index="0" model="ich9-uhci1"/>  

    <controller type="usb" index="0" model="ich9-uhci2"/>  

    <controller type="usb" index="0" model="ich9-uhci3"/>  

    <input type="tablet" bus="usb"/>  

    <input type="mouse" bus="ps2"/>  

    <graphics type="vnc" port="-1" autoport="yes" keymap="en-us" listen="0.0.0.0"/>  

    <channel type="unix"> 

      <source mode="bind" path="/var/lib/libvirt/qemu/nestedtest.org.qemu.guest_agent.0"/>  

      <target type="virtio" name="org.qemu.guest_agent.0"/> 

    </channel>  

    <interface type="bridge"> 

      <source bridge="em2eucabr"/>  

      <model type="virtio"/> 

    </interface>  

  </devices>  

  <qemu:commandline>

    <qemu:arg value='-machine'/>

    <qemu:arg value='smm=off'/>

  </qemu:commandline>

</domain>

#qemu-img info /home/nested_test_snap

image: nested_test_snap

file format: qcow2

virtual size: 50G (53687091200 bytes)

disk size: 51G

cluster_size: 65536

backing file: /home/nested_test_images

Format specific information:

    compat: 1.1

    lazy refcounts: false

libvirt of VM inside VM:

<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">  

  <name>i-test001</name>  

  <description>GCloud</description>  

  <uuid>2724aee4-e766-44e4-80b8-972a2cb32239</uuid>  

  <os> 

    <type>hvm</type>  

    <boot dev="hd"/> 

  </os>  

  <features> 

    <acpi/> 

  </features>  

  <clock offset="localtime"/>  

  <on_poweroff>destroy</on_poweroff>  

  <on_reboot>restart</on_reboot>  

  <on_crash>destroy</on_crash>  

  <vcpu current="1">1</vcpu>  

  <cpu mode="host-passthrough"/>  

  <memory>2097152</memory>  

  <currentMemory>2097152</currentMemory>  

  <devices> 

   <emulator>/root/qemu/x86_64-softmmu/qemu-system-x86_64</emulator> 

   <disk device="disk" type="file"> 

      <driver name="qemu" type="qcow2" cache="none"/>  

      <source file="/home/i-test001_snap.qcow2"/>  

      <target bus="virtio" dev="vda"/> 

    </disk>  

    <disk type="file" device="cdrom"> 

      <source dev=""/>  

      <target dev="hdc" bus="ide"/> 

    </disk>  

    <serial type="pty"> 

      <source path="/dev/pts/1"/>  

      <target port="0"/> 

    </serial>  

    <controller type="usb" index="0" model="ich9-ehci1"/>  

    <controller type="usb" index="0" model="ich9-uhci1"/>  

    <controller type="usb" index="0" model="ich9-uhci2"/>  

    <controller type="usb" index="0" model="ich9-uhci3"/>  

    <input type="tablet" bus="usb"/>  

    <input type="mouse" bus="ps2"/>  

    <graphics type="vnc" port="-1" autoport="yes" keymap="en-us" listen="0.0.0.0"/>  

    <channel type="unix">

      <source mode="bind" path="/var/lib/libvirt/qemu/i-test001.org.qemu.guest_agent.0"/>

      <target type="virtio" name="org.qemu.guest_agent.0"/>

    </channel>

  </devices>  

    <qemu:commandline>

    <qemu:arg value='-machine'/>

    <qemu:arg value='smm=off'/>

  </qemu:commandline>

</domain>

#qemu-img info /home/i-test001_snap.qcow2

image: i-test001_snap.qcow2

file format: qcow2

virtual size: 20G (21474836480 bytes)

disk size: 79M

cluster_size: 65536

backing file: /home/win2008_standard.raw

Format specific information:

    compat: 1.1

    lazy refcounts: false

I also tried 

<cpu mode='custom' match='exact'>

    <model fallback='allow'>core2duo</model>

    <feature policy='require' name='vmx'/>

  </cpu>

in host VM.

Is there any mistake?

Thanks,

John