[Octave-bug-tracker] [bug #53644] Possible Octave or Nouveau image plot bug (not sure which) discovered via ASAN

[Dan Sebald]

Follow-up Comment #5, bug #53644 (project octave):

Regarding Comment #4, after testing fairly extensively last night, I don't
think the multiple drawing of a newly created figure is the root cause of this
issue.  I'm going to guess that perhaps the multiple "rendering" comes about
from the OS window manager.  (I'm going to carefully use the word "rendering"
as specifically referring to the OpenGL actions and "redraw" as referring to
Octave running its plot code.)  That is, some of these window managers get
sort of fancy with what they are doing.  For example, I think Cinnamon will
render a new app window multiple times--small-to-big--as though the window is
appearing out of the ether or something.  It could be that Plasma is doing the
same, but the window is always the same size, and the randomness of how many
draws may come from the window manager drawing based on timing, e.g., every
150 ms for 1 s factoring in the rendering speed.  So, it could be that the OS
window manager creates the figure window and keeps rendering the thing while
Octave is still populating the figure.  First it is a blank window, i.e.,

xPlane: 1
yPlane: 1
zPlane: 0
xPlaneN: 0
yPlaneN: 0
zPlaneN: 1

and somewhere along the way it becomes a populated window, i.e.,

xPlane: 64.5
yPlane: 0.5
zPlane: 0
xPlaneN: 0.5
yPlaneN: 64.5
zPlaneN: 1
is2d: 1

In any case, what that multiple rendering does when first creating the Figure
is it brings the root cause to the core because of the following description. 
It appears to be the first rendering after an "image" rendering is causing the
ASAN abort.  That is, if I first generate

>> figure

then separately call

>> image

there is no ASAN abort, but from that point forward *anything* redrawn (not
necessarily an image) will cause an abort in the
opengl_renderer::draw_axes_planes() rendering.  It doesn't have to be another
image().  And the failure is consistently at that same location I first
described.

Now, it is only the image() routine that will setup the failure condition.  I
can run ploat(1:50), surface(), etc. and keep replotting...none of that
rendering fails.  It's only when I call image() then do following rendering,
whether it is another Octave plot action, or resizing the window, etc.

So, the root cause lies in the image() rendering code, and it is difficult to
find.  I've practically eliminated the draw_image() OpenGL-aimed code and the
issue still occurs.  I wonder if maybe there is a colormap or some other
ancillary issue unique to image plotting.  Maybe within the OpenGL driver
Octave is instructing a colormap that is slightly too big causing the heap out
of bounds.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?53644>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/