[Qemu-devel] Question about a particular trap generation scenario

[John]

Hi,

 

I am trying to run a program (Windows guest) that I protected with a software protection product.  The program is segfaulting when run and it appears to me that the problem lies in how Qemu is handling an anti debug measure that involves traps.

 

The anti-debug sequence is something like this:

 

1. Install exception handler (via Windows SEH)

2. Generate an INT3 (CC) trap

3. The exception handler does some sneaky stuff and sets the trap flag (TF), by modifying the EFLAGS value on the stack

4. Subsequent instructions will generate INT1 traps, invoking the exception handler repeatedly.

5. Another INT3 is then encountered, resetting the trap flag.

 

My question is:  How is the x86 supposed to handle step 5 when the second INT3 is executed AND the trap flag is set and does this differ from how will Qemu handle this scenario?

 

I saw a comment in exec_loop that Qemu will not handle multiple exceptions - does  the situation described above fall into this category?

 

 

Thanks in advance,

John

---

Looking for earth-friendly autos? 
 Browse Top Cars by "Green Rating" at Yahoo! Autos' Green Center.