[Octave-bug-tracker] [bug #39925] imagesc crash

[Rik]

Follow-up Comment #16, bug #39925 (project octave):

The proximate cause is line 2603 of gl-render.cc:


j0 += (xmin - im_xmin)/nor_dx + 1;


j0 is an integer and this calculation overflows producing a negative value for
j0.  Later, j0 is used as an index into an array which causes the segfault.

I still don't understand the root cause though.  It seems it shouldn't be
possible to reach here with the values of xmin and im_xmin.


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?39925>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/