Hello,
I'm currently testing various binaries under qemu linux user and went into a strange bug.
Here is the TLDR: is there a reason to allow VEX.L to be 1 when not supporting AVX instructions? Crashing with illegal op may save some time and headache to users.
And now for some context:
One of my test binaries had some AVX instructions and crashed in a weird way.
As I understand AVX is not supported, and so a Qemu crash should be expected.
However, in this instance, it's the guest that crashed, long after the offending AVX instruction.
The faulty instruction was a `vmovups ymmword ptr [rsp], ymm0`.
Looking into i386/translate.c, it seems that it is correctly decoded but its generation (see 'case 0x111' in `gen_sse`) is invalid.
Indeed, while the VEX prefix is correctly decoded, its VEX.L bit is never used during the instruction generation and is always assumed to be 0.
Therefore, the instruction generated is a `vmovups xmmword ptr [rsp], xmm0`, using a 128bits register instead of the orignal one using a 256bits register.
My understanding is that the VEX prefix and 256 bits registers where introduced by AVX. As Qemu does not support AVX, is there any kind of reason not to crash (illegalop) when VEX.L = 1?
Best regards,
Thanks,
Stevie