Seg fault with test_sparse.m

[Joe Koski]

on 8/17/06 3:30 PM, David Bateman at address@hidden wrote:

> Joe Koski wrote:
>> on 8/16/06 2:47 PM, David Bateman at address@hidden wrote:
>> 
>> 
>>> Joe Koski wrote:
>>> 
>>>> Hi all,
>>>> 
>>>> After getting a successful build of octave-2.9.7 on my Mac G5 with OS X
>>>> 10.4.7, Xcode tools 2.3 (gcc-4.0.1) and gfortran, I installed dejaGNU and
>>>> ran make check. I got one IO error, but things were going well until I hit
>>>> test_sparse. There I got a seg fault. I have attached the terminal output
>>>> and fntest.log output in case you want to see them.
>>>> 
>>>> I then moved to the test directory and ran octave. Here's what I got:
>>>> 
>>>> octave:1> test_sparse
>>>> writing test output to sptest.log
>>>>  
>>>> 
>>>>>>>>> /Tools/octave-2.9.7/test/test_sparse.m
>>>>>>>>>          
>>>> 
>>>> warning: Calculating Sparse Matrix Type
>>>> panic: Bus error -- stopping myself...
>>>> Bus error
>>>> Joe-Koskis-Computer:/Tools/octave-2.9.7/test jakoski$
>>>> 
>>>> I have also attached the sptest.log for your information.
>>>> 
>>>> To my untrained eye, the script seems to be running through quite a few
>>>> tests before the failure. Things are almost working.
>>>> 
>>>> Anyway, I'll be available to try things, but maybe it's time for someone
>>>> with sparse matrix knowledge (definitely not me) to try this on a Mac. I'll
>>>> be available to assist anyone who wants to try building octave-2.9.7 or
>>>> later on a Mac under OS X 10.4. I can cut many hours from the build process
>>>> with my experience from the last few days.
>>>> 
>>>> The other problem that I have is with the legend.m routine in octave-forge.
>>>> The problem is persistent with any script that uses legend. As I said, I
>>>> can
>>>> file a bug report, if it is an unknown problem.
>>>> 
>>>> Meanwhile, I'll go back to octave-2.1.73 for my routine stuff. Let me know
>>>> when or if I can help or try something.
>>>> 
>>>> As an aside, I've been around complicated software and complicated
>>>> engineering systems for over 40 years, and I have seldom seen things that
>>>> went cleanly from start to finish. There's always something, and the
>>>> problem
>>>> never seems to be the thing that you were worrying about.
>>>> 
>>>> Joe
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>  
>>> 
>>> I tried looking at this in June with Marius Schamschula, but couldn't
>>> identify the exact test causing the issues. Firstly, is UFSparse
>>> correctly installed? If so which version? In any case, I'd promised
>>> myself I'd steal my wife's powerbook for a few hours to test this out
>>> one of these days, though I'd be happy if some one else could identify
>>> exactly which test is failing. From your logs it seems that something like
>>> 
>>> sm = sprandn(8,8,0.2)
>>> fm = rand(8,8);
>>> am = sm .^ fm;
>>> am2 = am(find(am != 0)
>>> 
>>> might fail for your. If it does I have a chance of finding the problem
>>> without bothering my wife :-)
>>> 
>>> D.
>>> 
>> 
>> David,
>> 
>> You are so correct. Your test fails on my system:
>> 
>> Joe-Koskis-Computer:~/Codes/octave_test_files jakoski$ octave
>> GNU Octave, version 2.9.7 (powerpc-apple-darwin8.7.0).
>> Copyright (C) 2006 John W. Eaton.
>> This is free software; see the source code for copying conditions.
>> There is ABSOLUTELY NO WARRANTY; not even for MERCHANTIBILITY or
>> FITNESS FOR A PARTICULAR PURPOSE.  For details, type `warranty'.
>> 
>> Additional information about Octave is available at http://www.octave.org.
>> 
>> Please contribute if you find this software useful.
>> For more information, visit http://www.octave.org/help-wanted.html
>> 
>> Report bugs to <address@hidden> (but first, please read
>> http://www.octave.org/bugs.html to learn how to write a helpful report).
>> 
>> octave:1> sm = sprandn(8,8,0.2)
>> sm =
>> 
>> Compressed Column Sparse (rows = 8, cols = 8, nnz = 12)
>> 
>>   (1, 1) -> -1.2817
>>   (6, 1) ->  1.0076
>>   (7, 1) -> -0.65782
>>   (5, 2) -> -1.2952
>>   (7, 3) -> -0.66335
>>   (1, 4) -> -0.64771
>>   (4, 4) ->  0.63989
>>   (7, 4) -> -0.40588
>>   (8, 6) -> -0.82241
>>   (5, 8) ->  1.1687
>>   (6, 8) ->  1.4763
>>   (7, 8) ->  0.24860
>> 
>> octave:2> fm = rand(8,8);
>> octave:3> am = sm .^ fm;
>> panic: Bus error -- stopping myself...
>> 
>> My UFSparse came directly from the Florida website:
>> 
>>   http://www.cise.ufl.edu/research/sparse/umfpack/
>> 
>> Which shows in the README as version 5.0. Of course there are seven other
>> libraries that must also be built before all the externals are satisfied
>> (according to the octave configure script, anyway): AMD-2.0, COLAMD-2.5,
>> CCOLAMD-2.5, CHOLMOD-1.1.1, CXSparse-2.0.1, metis-4.0, and CAMD-2.1. Since
>> all these are in separate directories, my fear is that a header file in one
>> library doesn't match the same header in another library. Just a thought.
>> Unfortunately, none of the libraries are easy to identify without opening
>> each individual README file. Of course metis-4.0 comes from
>> 
>>   http://glaros.dtc.umn.edu/gkhome/views/metis
>> 
>> If you want me to rebuild and retest, I'm available. Just let me know. I
>> just installed the Apple Xcode-2.4 developer tools which were released about
>> a week ago. I haven't tried building anything with the new tools yet,
>> though. I'm trying to keep the latest version of my OS and Xcode tools on
>> the premise that it's the target configuration for octave.
>> 
>> Joe
>> 
> 
> Joe,
> 
> I have no idea why the existing code in sparse-xpow.cc is causing issues
> on big-endian Mac's. However, looking at the code I think it needs a
> major overhaul in any case as the existing code is much slower than it
> needs to be. In any case ".^" is a pretty stupid operator to use on a
> sparse matrix, so I don't see that it should be optimized too much. In
> any case, can you try the attached patch on your Mac and see if the
> short test I sent you now works. Can you also then run "make check"
> again and find the next place (if any) where there are issues :-)
> 
> D.
> 

David,

I applied the patch and did a complete rebuild (27 minutes with make -j2).
Things are much better. I did the rebuild because I wasn't sure if my new
Xcode-2.4 was totally compatible with the previous Xcode-2.3, and to make
sure I could still build octave with 2.4.

Now, no seg fault, instead "make check" gives

test_sparse.m .......................................... PASS 1156/1176 FAIL
20

Then, as before, I moved to /test and ran test_sparse directly from octave.
I'm forwarding the sptest.log to you separately for your examination. I'm
not familiar enough with the tests to understand the output.

On the simple test that you sent, it runs through as follows.

octave:1> sm = sprandn(8,8,0.2)
sm =

Compressed Column Sparse (rows = 8, cols = 8, nnz = 9)

  (5, 3) -> -1.9247
  (7, 3) ->  0.66021
  (4, 4) -> -1.0960
  (4, 5) -> -1.7105
  (5, 6) -> -0.86533
  (2, 7) ->  0.83712
  (5, 7) ->  0.74998
  (4, 8) ->  0.24879
  (6, 8) -> -0.16006

octave:2> fm = rand(8,8);
octave:3> am = sm .^ fm;
octave:4> am2 = am(find(am != 0))
am2 =

   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
       NaN
   1.00000
   0.83566
   1.00000
   1.00000
   1.00000
   1.00000
       NaN
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
       NaN
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
       NaN
   1.00000
   1.00000
   1.00000
   1.00000
   0.93082
   1.00000
   1.00000
   0.96124
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   0.28230
   1.00000
       NaN
   1.00000
   1.00000

octave:5> 

I guess my only question is whether the NaNs should be there.

I agree that a sparse matrix with each element to a different non-sparse
power (at least that's my interpretation of the test that caused the
original problem) is not often done, especially by me.

If we wait a couple of years, most Macs will be little-endian, and this
problem will be history, but there are always a few PowerPCs, Irixes, etc.
out there.

Let me know what to try next. I'm available for testing.

Joe