Re: Seg fault with test_sparse.m

[David Bateman]

Joe Koski wrote:
> on 8/16/06 2:47 PM, David Bateman at address@hidden wrote:
> 
> 
>>Joe Koski wrote:
>>
>>>Hi all,
>>>
>>>After getting a successful build of octave-2.9.7 on my Mac G5 with OS X
>>>10.4.7, Xcode tools 2.3 (gcc-4.0.1) and gfortran, I installed dejaGNU and
>>>ran make check. I got one IO error, but things were going well until I hit
>>>test_sparse. There I got a seg fault. I have attached the terminal output
>>>and fntest.log output in case you want to see them.
>>>
>>>I then moved to the test directory and ran octave. Here's what I got:
>>>
>>>octave:1> test_sparse
>>>writing test output to sptest.log
>>>  
>>>
>>>>>>>>/Tools/octave-2.9.7/test/test_sparse.m
>>>>>>>>           
>>>
>>>warning: Calculating Sparse Matrix Type
>>>panic: Bus error -- stopping myself...
>>>Bus error
>>>Joe-Koskis-Computer:/Tools/octave-2.9.7/test jakoski$
>>>
>>>I have also attached the sptest.log for your information.
>>>
>>>To my untrained eye, the script seems to be running through quite a few
>>>tests before the failure. Things are almost working.
>>>
>>>Anyway, I'll be available to try things, but maybe it's time for someone
>>>with sparse matrix knowledge (definitely not me) to try this on a Mac. I'll
>>>be available to assist anyone who wants to try building octave-2.9.7 or
>>>later on a Mac under OS X 10.4. I can cut many hours from the build process
>>>with my experience from the last few days.
>>>
>>>The other problem that I have is with the legend.m routine in octave-forge.
>>>The problem is persistent with any script that uses legend. As I said, I can
>>>file a bug report, if it is an unknown problem.
>>>
>>>Meanwhile, I'll go back to octave-2.1.73 for my routine stuff. Let me know
>>>when or if I can help or try something.
>>>
>>>As an aside, I've been around complicated software and complicated
>>>engineering systems for over 40 years, and I have seldom seen things that
>>>went cleanly from start to finish. There's always something, and the problem
>>>never seems to be the thing that you were worrying about.
>>>
>>>Joe
>>>
>>>
>>>
>>> 
>>>
>>>  
>>
>>I tried looking at this in June with Marius Schamschula, but couldn't
>>identify the exact test causing the issues. Firstly, is UFSparse
>>correctly installed? If so which version? In any case, I'd promised
>>myself I'd steal my wife's powerbook for a few hours to test this out
>>one of these days, though I'd be happy if some one else could identify
>>exactly which test is failing. From your logs it seems that something like
>>
>>sm = sprandn(8,8,0.2)
>>fm = rand(8,8);
>>am = sm .^ fm;
>>am2 = am(find(am != 0)
>>
>>might fail for your. If it does I have a chance of finding the problem
>>without bothering my wife :-)
>>
>>D.
>>
> 
> David,
> 
> You are so correct. Your test fails on my system:
> 
> Joe-Koskis-Computer:~/Codes/octave_test_files jakoski$ octave
> GNU Octave, version 2.9.7 (powerpc-apple-darwin8.7.0).
> Copyright (C) 2006 John W. Eaton.
> This is free software; see the source code for copying conditions.
> There is ABSOLUTELY NO WARRANTY; not even for MERCHANTIBILITY or
> FITNESS FOR A PARTICULAR PURPOSE.  For details, type `warranty'.
> 
> Additional information about Octave is available at http://www.octave.org.
> 
> Please contribute if you find this software useful.
> For more information, visit http://www.octave.org/help-wanted.html
> 
> Report bugs to <address@hidden> (but first, please read
> http://www.octave.org/bugs.html to learn how to write a helpful report).
> 
> octave:1> sm = sprandn(8,8,0.2)
> sm =
> 
> Compressed Column Sparse (rows = 8, cols = 8, nnz = 12)
> 
>   (1, 1) -> -1.2817
>   (6, 1) ->  1.0076
>   (7, 1) -> -0.65782
>   (5, 2) -> -1.2952
>   (7, 3) -> -0.66335
>   (1, 4) -> -0.64771
>   (4, 4) ->  0.63989
>   (7, 4) -> -0.40588
>   (8, 6) -> -0.82241
>   (5, 8) ->  1.1687
>   (6, 8) ->  1.4763
>   (7, 8) ->  0.24860
> 
> octave:2> fm = rand(8,8);
> octave:3> am = sm .^ fm;
> panic: Bus error -- stopping myself...
> 
> My UFSparse came directly from the Florida website:
> 
>   http://www.cise.ufl.edu/research/sparse/umfpack/
> 
> Which shows in the README as version 5.0. Of course there are seven other
> libraries that must also be built before all the externals are satisfied
> (according to the octave configure script, anyway): AMD-2.0, COLAMD-2.5,
> CCOLAMD-2.5, CHOLMOD-1.1.1, CXSparse-2.0.1, metis-4.0, and CAMD-2.1. Since
> all these are in separate directories, my fear is that a header file in one
> library doesn't match the same header in another library. Just a thought.
> Unfortunately, none of the libraries are easy to identify without opening
> each individual README file. Of course metis-4.0 comes from
> 
>   http://glaros.dtc.umn.edu/gkhome/views/metis
> 
> If you want me to rebuild and retest, I'm available. Just let me know. I
> just installed the Apple Xcode-2.4 developer tools which were released about
> a week ago. I haven't tried building anything with the new tools yet,
> though. I'm trying to keep the latest version of my OS and Xcode tools on
> the premise that it's the target configuration for octave.
> 
> Joe
> 

Joe,

I have no idea why the existing code in sparse-xpow.cc is causing issues
on big-endian Mac's. However, looking at the code I think it needs a
major overhaul in any case as the existing code is much slower than it
needs to be. In any case ".^" is a pretty stupid operator to use on a
sparse matrix, so I don't see that it should be optimized too much. In
any case, can you try the attached patch on your Mac and see if the
short test I sent you now works. Can you also then run "make check"
again and find the next place (if any) where there are issues :-)

D.


Index: src/sparse-xpow.cc
===================================================================
RCS file: /cvs/octave/src/sparse-xpow.cc,v
retrieving revision 1.8
diff -c -r1.8 sparse-xpow.cc
*** src/sparse-xpow.cc  3 May 2006 19:32:47 -0000       1.8
--- src/sparse-xpow.cc  17 Aug 2006 21:26:45 -0000
***************
*** 406,480 ****
  
    int convert_to_complex = 0;
    for (octave_idx_type j = 0; j < nc; j++)
!     for (octave_idx_type i = 0; i < nr; i++)
        {
!       OCTAVE_QUIT;
!       double atmp = a (i, j);
!       double btmp = b (i, j);
!       if (atmp < 0.0 && static_cast<int> (btmp) != btmp)
          {
!           convert_to_complex = 1;
!           goto done;
          }
        }
  
  done:
  
!   octave_idx_type nel = 0;
!   for (octave_idx_type j = 0; j < nc; j++) 
!     for (octave_idx_type i = 0; i < nr; i++)
!       if (!(a.elem (i, j) == 0. && b.elem (i, j) != 0.))
!       nel++;
  
    if (convert_to_complex)
      {
!       SparseComplexMatrix complex_result (nr, nc, nel);
  
-       octave_idx_type ii = 0;
-       complex_result.cidx(0) = 0;
        for (octave_idx_type j = 0; j < nc; j++)
        {
!         for (octave_idx_type i = 0; i < nr; i++)
            {
              OCTAVE_QUIT;
!             Complex atmp (a (i, j));
!             Complex btmp (b (i, j));
!             Complex tmp =  pow (atmp, btmp);
!             if (tmp != 0.)
!               {
!                 complex_result.data (ii) = tmp;
!                 complex_result.ridx (ii++) = i;
!               }
            }
-         complex_result.cidx (j+1) = ii;
        }
!       complex_result.maybe_compress ();
! 
        retval = complex_result;
      }
    else
      {
!       SparseMatrix result (nr, nc, nel);
!       octave_idx_type ii = 0;
  
-       result.cidx (0) = 0;
        for (octave_idx_type j = 0; j < nc; j++)
        {
!         for (octave_idx_type i = 0; i < nr; i++)
            {
              OCTAVE_QUIT;
!             double tmp = pow (a (i, j), b (i, j));
!             if (tmp != 0.)
!               {
!                 result.data (ii) = tmp;
!                 result.ridx (ii++) = i;
!               }
            }
-         result.cidx (j+1) = ii;
        }
! 
!       result.maybe_compress ();
! 
        retval = result;
      }
  
--- 406,460 ----
  
    int convert_to_complex = 0;
    for (octave_idx_type j = 0; j < nc; j++)
!     for (octave_idx_type i = a.cidx(j); i < a.cidx(j+1); i++)
        {
!       if (a.data(i) < 0.0)
          {
!           double btmp = b (a.ridx(i), j);
!           if (static_cast<int> (btmp) != btmp)
!             {
!               convert_to_complex = 1;
!               goto done;
!             }
          }
        }
  
  done:
  
!   // This is a dumb operator for sparse matrices anyway, and there is
!   // no sensible way to handle the 0.^0 versus the 0.^x cases. Therefore
!   // allocate a full matrix filled for the 0.^0 case and shrink it later
!   // as needed
  
    if (convert_to_complex)
      {
!       SparseComplexMatrix complex_result (nr, nc, Complex(1.0, 0.0));
  
        for (octave_idx_type j = 0; j < nc; j++)
        {
!         for (octave_idx_type i = a.cidx(j); i < a.cidx(j+1); i++)
            {
              OCTAVE_QUIT;
!             complex_result.xelem(a.ridx(i), j) = pow (a.data(i), 
!                                                       b (a.ridx(i), j));
            }
        }
!       complex_result.maybe_compress (true);
        retval = complex_result;
      }
    else
      {
!       SparseMatrix result (nr, nc, 1.0);
  
        for (octave_idx_type j = 0; j < nc; j++)
        {
!         for (octave_idx_type i = a.cidx(j); i < a.cidx(j+1); i++)
            {
              OCTAVE_QUIT;
!             result.xelem(a.ridx(i), j) = pow (a.data(i), b (a.ridx(i), j));
            }
        }
!       result.maybe_compress (true);
        retval = result;
      }
  
***************
*** 525,556 ****
        return octave_value ();
      }
  
!   octave_idx_type nel = 0;
!   for (octave_idx_type j = 0; j < nc; j++) 
!     for (octave_idx_type i = 0; i < nr; i++)
!       if (!(a.elem (i, j) == 0. && b.elem (i, j) != 0.))
!       nel++;
! 
!   SparseComplexMatrix result (nr, nc, nel);
!   octave_idx_type ii = 0;
! 
!   result.cidx(0) = 0;
    for (octave_idx_type j = 0; j < nc; j++)
      {
!       for (octave_idx_type i = 0; i < nr; i++)
        {
          OCTAVE_QUIT;
!         Complex tmp = pow (Complex (a (i, j)), b (i, j));
!         if (tmp != 0.)
!           {
!             result.data (ii) = tmp; 
!             result.ridx (ii++) = i; 
!           }
        }
-       result.cidx (j+1) = ii;
      }
  
!   result.maybe_compress ();
  
    return result;
  }
--- 505,521 ----
        return octave_value ();
      }
  
!   SparseComplexMatrix result (nr, nc, Complex(1.0, 0.0));
    for (octave_idx_type j = 0; j < nc; j++)
      {
!       for (octave_idx_type i = a.cidx(j); i < a.cidx(j+1); i++)
        {
          OCTAVE_QUIT;
!         result.xelem(a.ridx(i), j) = pow (a.data(i), b (a.ridx(i), j));
        }
      }
  
!   result.maybe_compress (true);
  
    return result;
  }
***************
*** 680,717 ****
        return octave_value ();
      }
  
!   octave_idx_type nel = 0;
!   for (octave_idx_type j = 0; j < nc; j++) 
!     for (octave_idx_type i = 0; i < nr; i++)
!       if (!(a.elem (i, j) == 0. && b.elem (i, j) != 0.))
!       nel++;
! 
!   SparseComplexMatrix result (nr, nc, nel);
!   octave_idx_type ii = 0;
! 
!   result.cidx (0) = 0;
    for (octave_idx_type j = 0; j < nc; j++)
      {
!       for (octave_idx_type i = 0; i < nr; i++)
        {
          OCTAVE_QUIT;
!         double btmp = b (i, j);
          Complex tmp;
  
          if (xisint (btmp))
!           tmp = pow (a (i, j), static_cast<int> (btmp));
          else
!           tmp = pow (a (i, j), btmp);
!         if (tmp != 0.)
!           {
!             result.data (ii) = tmp; 
!             result.ridx (ii++) = i; 
!           }
        }
-       result.cidx (j+1) = ii;
      }
  
!   result.maybe_compress ();
  
    return result;
  }
--- 645,668 ----
        return octave_value ();
      }
  
!   SparseComplexMatrix result (nr, nc, Complex(1.0, 0.0));
    for (octave_idx_type j = 0; j < nc; j++)
      {
!       for (octave_idx_type i = a.cidx(j); i < a.cidx(j+1); i++)
        {
          OCTAVE_QUIT;
!         double btmp = b (a.ridx(i), j);
          Complex tmp;
  
          if (xisint (btmp))
!           result.xelem(a.ridx(i), j) = pow (a.data (i), 
!                                             static_cast<int> (btmp));
          else
!           result.xelem(a.ridx(i), j) = pow (a.data (i), btmp);
        }
      }
  
!   result.maybe_compress (true);
  
    return result;
  }
***************
*** 762,790 ****
        return octave_value ();
      }
  
!   octave_idx_type nel = 0;
!   for (octave_idx_type j = 0; j < nc; j++) 
!     for (octave_idx_type i = 0; i < nr; i++)
!       if (!(a.elem (i, j) == 0. && b.elem (i, j) != 0.))
!       nel++;
! 
!   SparseComplexMatrix result (nr, nc, nel);
!   octave_idx_type ii = 0;
! 
!   result.cidx (0) = 0;
    for (octave_idx_type j = 0; j < nc; j++) 
      {
!       for (octave_idx_type i = 0; i < nr; i++)
        {
          OCTAVE_QUIT;
!         Complex tmp = pow (a (i, j), b (i, j));
!         if (tmp != 0.)
!           {
!             result.data (ii) = tmp;
!             result.ridx (ii++) = i;
!           }
        }
-       result.cidx (j+1) = ii;
      }
    result.maybe_compress (true);
  
--- 713,726 ----
        return octave_value ();
      }
  
!   SparseComplexMatrix result (nr, nc, Complex(1.0, 0.0));
    for (octave_idx_type j = 0; j < nc; j++) 
      {
!       for (octave_idx_type i = a.cidx(j); i < a.cidx(j+1); i++)
        {
          OCTAVE_QUIT;
!         result.xelem(a.ridx(i), j) = pow (a.data (i), b (a.ridx(i), j));
        }
      }
    result.maybe_compress (true);