[Octave-bug-tracker] [bug #42850] Reshape on sparse matrix leads to segfault

[Eduardo Ramos Fernández]

Follow-up Comment #2, bug #42850 (project octave):

I have been taking a look at the problem and the issue is located within the
reshape method of Sparse<T> class. In Sparse.cc around line 845:



 for (octave_idx_type i = 0; i < old_nc; i++) 
            for (octave_idx_type j = cidx (i); j < cidx (i+1); j++) 
              {    
                *octave_idx_type tmp = i * old_nr + ridx (j);*
                octave_idx_type ii = tmp % new_nr;
                octave_idx_type jj = (tmp - ii) / new_nr;
                for (octave_idx_type k = kk; k < jj; k++) 
                  retval.xcidx (k+1) = j; 
                kk = jj;
                retval.xdata (j) = data (j); 
                retval.xridx (j) = ii;
              }    



With an example similar to the David's one, a segfault is obtained just by the
time is executed.

b = logical(speye(1000000)); 
c = reshape(b,200000,5000000);

The problem is the variable *tmp*, that can become negative due to integer
overflowing. In the example old_nr = 1000000 and i take values up to
old_nc=1000000. So their product can be greater than 2^30 = 1.0737e+09. 

I don't really know how to fix this in a proper way without using 64 bits.
Maybe other knows a quick fix.




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?42850>

_______________________________________________
  Mensaje enviado vía/por Savannah
  http://savannah.gnu.org/