Re: [Qemu-devel] [PATCH] ui/vnc.c: Fix crash with VNC

[Peter Maydell]

On 1 November 2012 21:06, Gerhard Wiesinger <address@hidden> wrote:
> Fix crash with VNC under NT 4.0 and VMWare VGA and window which is outside
> of the visible area.
>
> Backtrace:
> #0  set_bit (addr=<optimized out>, nr=-3) at ./bitops.h:122
> #1  vnc_dpy_update (ds=<optimized out>, x=-48, y=145, w=57, h=161) at
> ui/vnc.c:452
> #2  0x00007f1ce057e2ec in dpy_update (s=0x7f1ce1c8c880, h=16, w=66, y=145,
> x=-57) at ./console.h:242
> #3  vmsvga_update_rect (h=16, w=66, y=145, x=-57, s=0x7f1ce1cb3dd0) at
> hw/vmware_vga.c:324
> #4  vmsvga_update_rect_flush (s=0x7f1ce1cb3dd0) at hw/vmware_vga.c:357
> #5  vmsvga_update_display (opaque=0x7f1ce1cb3dd0) at hw/vmware_vga.c:960
> #6  0x00007f1ce05f0b37 in vnc_refresh (opaque=0x7f1cd8526010) at
> ui/vnc.c:2590
> #7  0x00007f1ce05c002b in qemu_run_timers (clock=0x7f1ce1c4f910) at
> qemu-timer.c:392
> #8  qemu_run_timers (clock=0x7f1ce1c4f910) at qemu-timer.c:373
> #9  0x00007f1ce05c028d in qemu_run_all_timers () at qemu-timer.c:449
> #10 0x00007f1ce058f2ee in main_loop_wait (nonblocking=<optimized out>) at
> main-loop.c:502
> #11 0x00007f1ce047acb3 in main_loop () at vl.c:1655
> #12 main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>)
> at vl.c:3826
>
> Signed-off-by: Gerhard Wiesinger <address@hidden>
> ---
>  ui/vnc.c | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/ui/vnc.c b/ui/vnc.c
> index 7c120e6..ae6d819 100644
> --- a/ui/vnc.c
> +++ b/ui/vnc.c
> @@ -453,6 +453,11 @@ static void vnc_dpy_update(DisplayState *ds, int x, int
> y, int w, int h)
>      w = MIN(x + w, width) - x;
>      h = MIN(h, height);
>
> +    x = MAX(x, 0);
> +    y = MAX(y, 0);
> +    w = MAX(w, 0);
> +    h = MAX(h, 0);
> +
>      for (; y < h; y++)
>          for (i = 0; i < w; i += 16)
>              set_bit((x + i) / 16, s->dirty[y]);
> --
> 1.7.11.7

I think this is fixing this at the wrong level. Either we
should require that drivers (in this case vmware_vga.c)
must not call dpy_gfx_update() with out of range values,
or we should do the clipping in the console.c layer, but
I don't think requiring every UI backend to clip is the
right thing. Anthony?

-- PMM