Re: [Qemu-devel] QEMU, self-modifying code, and Windows 7 64-bit (no KVM)

[Hulin, Patrick - 0559 - MITLL]

On Aug 15, 2014, at 4:48 PM, Paolo Bonzini <address@hidden> wrote:

> Il 13/08/2014 20:36, Hulin, Patrick - 0559 - MITLL ha scritto:
>> Hi QEMU devs,
>> 
>> QEMU 2.10 does not currently run Windows 7 64-bit without KVM. There
>> have been a few threads about this over the past few years (such as
>> https://bugs.launchpad.net/qemu/+bug/921208 and
>> http://lists.gnu.org/archive/html/qemu-devel/2012-09/msg02603.html),
>> but the problem was never resolved. I think I've identified the
>> cause, but I am not sure what the correct way to fix it is. I'm
>> working on PANDA, a set of analysis extensions to QEMU
>> (github.com/moyix/panda) and I'd really like to be able to use our
>> analyses on Windows 7 64-bit.
>> 
>> There are two issues right now. The first is that QEMU is missing a
>> CPUID bit (for debug extensions, CPUID_DE) because the feature isn't
>> implemented in QEMU. This can easily be hacked around by just
>> enabling the bit, but I imagine you all aren't excited about
>> advertising features that don't exist.
> 
> Not too worried about it either.
> 
> The two aspects of CPUID.DE are: 1) support for CR4.DE and raising an
> exception for DR4 and DR5 access; 2) I/O breakpoints.  Now, QEMU always
> raises the exception even if CR4.DE=0, and doesn't complain if you set
> bits of CR4 that ought to be reserved according to CPUID.  And I/O
> breakpoints aren't that hard to implement.  Having a full implementation
> of CPUID.DE wouldn't be hard, but I wouldn't have much problems with
> just setting the bit in TCG_FEATURES and noting that it is only
> partially implemented.
> 
>> The second issue is that both
>> the installer and the OS itself fail with blue screens of
>> DRIVER_IRQL_NOT_LESS_OR_EQUAL or KMODE_EXCEPTION_NOT_HANDLED (due to
>> illegal instruction). This is a little trickier.
> 
> Indeed.  Thanks for debugging it.
> 
>> Before executing a translation block, QEMU write-protects (using
>> host MMU features) the _host_ page that contains the section of guest 
>> memory on which the guest TB code lives.
> 
> Are you sure?  My knowledge of this code is only cursory, but I think
> that only applies to user-mode emulation.  System emulation uses softmmu
> exclusively to trap such writes (TLB_NOTDIRTY or something like that).

Certainly possible. I don’t really understand this code either.

> 
>> In this case, the write is 8 bytes and unaligned, so it gets split
>> into 8 single-byte writes. In stock QEMU, these writes are done in
>> reverse order (see the loop in softmmu_template.h, line 402). The
>> third decryption xor from Kernel Patch Protection should hit 4 bytes
>> that are in the current TB and 4 bytes in the TB afterwards in linear
>> order. Since this happens in reverse order, and the last 4 bytes of
>> the write do not intersect the current TB, those writes happen
>> successfully and QEMU's memory is modified. The 4th byte in linear
>> order (the 5th in temporal order) then triggers the
>> current_tb_modified flag and cpu_restore_state, longjmp'ing out.
>> 
>> I am not sure how to fix this issue. For now, in our tool, PANDA, we
>> have just reversed the order of the loop. But that change will fail
>> in any situation in which the write happens off the front end of the
>> TB and then the self-modifying code loops back to the previous TB.
>> This modification enables Windows 7 x64 to run successfully without
>> KVM, which is all we really need for our purposes.
> 
> Would it work to just call tb_invalidate_phys_page_range before the
> helper_ret_stb loop?
> 
> Paolo

Maybe. I think there’s another issue, which is that QEMU’s ending up in the I/O 
read/write code instead of the normal memory RW. This could be QEMU messing up, 
it could be PatchGuard doing something weird, or it could be me 
misunderstanding what’s going on. I never really figured out how the control 
flow works here.

I’m going to look at this more next week—I’ll see if I have more info then.