Re: [Qemu-devel] [PATCH v3] hw/display/qxl: fix signed to unsigned comparison

[Peter Maydell]

On 20 January 2014 14:20, Alon Levy <address@hidden> wrote:
> Several small signedness / overflow corrections to qxl_create_guest_primary:
> 1. use 64 bit unsigned for size to avoid overflow possible from two 32
> bit multiplicants.
> 2. correct sign for requested_height
> 3. add a more verbose error message when setting guest bug state (which
> causes a complete guess blackout until reset, so it helps if it is
> verbose).
>
> Signed-off-by: Alon Levy <address@hidden>
> ---
>  hw/display/qxl.c | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
>
> diff --git a/hw/display/qxl.c b/hw/display/qxl.c
> index e4f172e..ba752b5 100644
> --- a/hw/display/qxl.c
> +++ b/hw/display/qxl.c
> @@ -1360,14 +1360,15 @@ static void qxl_create_guest_primary(PCIQXLDevice 
> *qxl, int loadvm,
>  {
>      QXLDevSurfaceCreate surface;
>      QXLSurfaceCreate *sc = &qxl->guest_primary.surface;
> -    int size;
> -    int requested_height = le32_to_cpu(sc->height);
> +    uint64_t size;
> +    uint32_t requested_height = le32_to_cpu(sc->height);
>      int requested_stride = le32_to_cpu(sc->stride);
>
>      size = abs(requested_stride) * requested_height;

This is still doing a 32 x 32 multiply. You need to
make at least one side of the multiply use a 64 bit type
(via cast or by having it be a 64 bit type).

You're also still potentially passing minimum-integer
into abs. That should probably just be caught before we
do the abs() since it will always mean the size is too
large.

thanks
-- PMM