Re: Please build the JIT branch

[Ben Abbott]

On Jul 20, 2012, at 10:14 AM, Max Brister wrote:

> On Tue, Jul 17, 2012 at 6:25 AM, Ben Abbott <address@hidden> wrote:
>> 
>> On Jul 16, 2012, at 4:55 PM, Max Brister wrote:
>> 
>>> On Fri, Jul 13, 2012 at 6:27 AM, Ben Abbott <address@hidden> wrote:
>>>> On Jul 12, 2012, at 11:13 PM, Max Brister wrote:
>>>> 
>>>>> On Thu, Jul 12, 2012 at 8:18 PM, Ben Abbott <address@hidden> wrote:
>>>>>> 
>>>>>> On Jul 12, 2012, at 7:52 PM, Ben Abbott wrote:
>>>>>> 
>>>>>>> On Jul 10, 2012, at 6:00 PM, Max Brister wrote:
>>>>>>> 
>>>>>>>> JIT is still pretty limited, it will not compile loops with any
>>>>>>>> function calls, even builtin functions (except for sin, cos, and exp).
>>>>>>>> It also only supports linear matrix indexing. For an example of a
>>>>>>>> function which can be compiled, see
>>>>>>>> http://jit-octave.blogspot.com/2012/06/realistic-test.html.
>>>>>>>> 
>>>>>>>> Max Brister
>>>>>>> 
>>>>>>> Using macports on MacOS 10.7, I did a quick build (without worrying 
>>>>>>> about LLVM)
>>>>>>> 
>>>>>>> With JIT
>>>>>>> 
>>>>>>>    A = gen_test (1000000);
>>>>>>>    K = 500;
>>>>>>>    Vectorized: 1.274 sec
>>>>>>>    Loopy: 4.875 sec
>>>>>>> 
>>>>>>> With 3.7.0+
>>>>>>> 
>>>>>>>    A = gen_test (1000000);
>>>>>>>    K = 500;
>>>>>>>    Vectorized: 5.944
>>>>>>>    Loopy: 16.063
>>>>>>> 
>>>>>>> I'll try again with LLVM_CONFIG=/opt/local/bin/llvm-config-mp-3.0
>>>>>>> 
>>>>>>> Ben
>>>>> 
>>>>> It's odd that there was any change at all between the JIT branch
>>>>> (without being able to find llvm) and 3.7.0+. The JIT branch compiled
>>>>> without llvm should be mostly the same as 3.7.0+. Almost all of the
>>>>> code I have added is inside
>>>>> #ifdef LLVM_FOUND
>>>>> ....
>>>>> #endif // LLVM_FOUND
>>>>> 
>>>>> Maybe this is a fluke?
>>>> 
>>>> I didn't check, but I have an automated backup that may have been running 
>>>> with 3.7.0
>>>> 
>>>>>> With LLVM_CONFIG set to the above, I see ...
>>>>>> 
>>>>>> octave(56971,0x7fff70d62960) malloc: *** error for object 0x106a4c620: 
>>>>>> pointer being freed was not allocated
>>>>>> *** set a breakpoint in malloc_error_break to debug
>>>>>> 
>>>>>> Program received signal SIGABRT, Aborted.
>>>>>> 0x00007fff850fece2 in __pthread_kill ()
>>>>>> (gdb) bt
>>>>>> #0  0x00007fff850fece2 in __pthread_kill ()
>>>>>> #1  0x00007fff856e57d2 in pthread_kill ()
>>>>>> #2  0x00007fff856d6a7a in abort ()
>>>>>> #3  0x00007fff8573584c in free ()
>>>>>> #4  0x00000001069e2685 in std::string::assign ()
>>>>>> #5  0x0000000100b4297c in global constructors keyed to a () at 
>>>>>> basic_string.h:500
>>>>>> #6  0x00007fff5fc0fda6 in 
>>>>>> __dyld__ZN16ImageLoaderMachO18doModInitFunctionsERKN11ImageLoader11LinkContextE
>>>>>>  ()
>>>>>> #7  0x00007fff5fc0faf2 in 
>>>>>> __dyld__ZN16ImageLoaderMachO16doInitializationERKN11ImageLoader11LinkContextE
>>>>>>  ()
>>>>>> #8  0x00007fff5fc0d2e4 in 
>>>>>> __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE
>>>>>>  ()
>>>>>> #9  0x00007fff5fc0d27d in 
>>>>>> __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE
>>>>>>  ()
>>>>>> #10 0x00007fff5fc0e0b7 in 
>>>>>> __dyld__ZN11ImageLoader15runInitializersERKNS_11LinkContextERNS_21InitializerTimingListE
>>>>>>  ()
>>>>>> #11 0x00007fff5fc034dd in __dyld__ZN4dyld24initializeMainExecutableEv ()
>>>>>> #12 0x00007fff5fc0760b in 
>>>>>> __dyld__ZN4dyld5_mainEPK12macho_headermiPPKcS5_S5_ ()
>>>>>> #13 0x00007fff5fc01059 in __dyld__dyld_start ()
>>>>>> (gdb)
>>>>>> 
>>>>>> Ben
>>>>> 
>>>>> The stack trace is not very clear, but it looks like something bad is
>>>>> happening during static initialization. I should probably get rid of
>>>>> these static variables anyways. Can you try the attached patch? If I
>>>>> am right it should fix problem.
>>>>> 
>>>>> I might be a bit slow to respond the next few days. I have to prepare
>>>>> for and get to OctConf!
>>>>> 
>>>>> Max Brister
>>>>> <static_init.patch>
>>>> 
>>>> I don't see any change
>>>> 
>>>> done
>>>> octave(88684,0x7fff70d62960) malloc: *** error for object 0x106a4b620: 
>>>> pointer being freed was not allocated
>>>> *** set a breakpoint in malloc_error_break to debug
>>>> 
>>>> Program received signal SIGABRT, Aborted.
>>>> 0x00007fff850fece2 in __pthread_kill ()
>>>> (gdb) bt
>>>> #0  0x00007fff850fece2 in __pthread_kill ()
>>>> #1  0x00007fff856e57d2 in pthread_kill ()
>>>> #2  0x00007fff856d6a7a in abort ()
>>>> #3  0x00007fff8573584c in free ()
>>>> #4  0x00000001069e1685 in std::string::assign ()
>>>> #5  0x0000000100b4287c in global constructors keyed to a () at 
>>>> basic_string.h:500
>>>> #6  0x00007fff5fc0fda6 in 
>>>> __dyld__ZN16ImageLoaderMachO18doModInitFunctionsERKN11ImageLoader11LinkContextE
>>>>  ()
>>>> #7  0x00007fff5fc0faf2 in 
>>>> __dyld__ZN16ImageLoaderMachO16doInitializationERKN11ImageLoader11LinkContextE
>>>>  ()
>>>> #8  0x00007fff5fc0d2e4 in 
>>>> __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE
>>>>  ()
>>>> #9  0x00007fff5fc0d27d in 
>>>> __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE
>>>>  ()
>>>> #10 0x00007fff5fc0e0b7 in 
>>>> __dyld__ZN11ImageLoader15runInitializersERKNS_11LinkContextERNS_21InitializerTimingListE
>>>>  ()
>>>> #11 0x00007fff5fc034dd in __dyld__ZN4dyld24initializeMainExecutableEv ()
>>>> #12 0x00007fff5fc0760b in 
>>>> __dyld__ZN4dyld5_mainEPK12macho_headermiPPKcS5_S5_ ()
>>>> #13 0x00007fff5fc01059 in __dyld__dyld_start ()
>>>> (gdb)
>>> 
>>> I'm not sure what is happening here. If the statics I'm initializing
>>> do not cause any issue and Octave runs by its self, I wonder if this
>>> problem is caused by llvm? Does just linking llvm into Octave without
>>> using jit at all cause this problem?
>>> 
>>> Max Brister
>> 
>> What is the best way for me to test that? Do I simply add the part below to 
>> ./configure when building the developer's sources from Savannah?
>> 
>>        LLVM_CONFIG=/opt/local/bin/llvm-config-mp-3.0
>> 
>> Ben
> 
> Sorry for the delay. That approach will not work, as the developers
> source from savanah does not search for llvm. I have attached a patch
> that changes configure.ac to disable HAVE_LLVM (even when LLVM is
> found). Then it call llvm::getGlobalContext () in octave_main. This
> should show us if LLVM is an issue, or it is my code.
> 
> Max Brister
> <llvm_check.patch>

I've been having trouble running configure on the default branch since JIT was 
merged.  WIth Carlo's recent changes, I'm able to configure again! I've tried 
three cases.

        (1) Building without LLVM_CONFIG results in a working octave.

        (2) Building with LLVM_CONFIG and without your patch (llvm_check.patch) 
 crashes.

        (3) Building with LLVM_CONFIG and with your patch (llvm_check.patch) 
also crashes.

I'm unfamiliar with LLVM. I have several versions of gcc development tools 
installed on my Mac (gcc-4.2, gcc-4.4, gcc-4.5, and gcc-4.6).  Might LLVM be 
sensitive to compiler choice?

Also, I've been trying to build an Octave App-bundle for MacOS, and have been 
having problems running the result on MacOS 10.6 (I'm using 10.7).

        terminate called after throwing an instance of 
'__gnu_cxx::__concurrence_lock_error'
        what():  __gnu_cxx::__concurrence_lock_error

This error is often reported to be related to -pthreads, which is mentioned in 
the backtrace I get when using LLVM.  Perhaps my build process has some needs 
some adjustment?  Do you (anyone?) see something suspicious in what I have 
below?

./autogen.sh

VER="4.5"
THREADS="-D_THREAD_SAFE -pthread"
export PREFIX=/usr/local
export CC=/opt/local/bin/gcc-mp-$VER
export CXX=/opt/local/bin/g++-mp-$VER
export CXXCPP="/opt/local/bin/g++-mp-$VER -E"
export F77=/opt/local/bin/gfortran-mp-$VER
export FC=/opt/local/bin/gfortran-mp-$VER
export CXXFLAGS="-pipe -O0 -g -m64 -ggdb3 -gstabs $THREADS"
export FFLAGS="$CXXFLAGS"
export CFLAGS="$FFLAGS -lstdc++"
export LDFLAGS="$THREADS -L/opt/local/lib"
export CPPFLAGS="$THREADS -I/opt/local/include"
export BLAS_LIBS="-lcblas -lf77blas -latlas"
export LAPACK_LIBS=-llapack

./configure --prefix="$PREFIX" --with-framework-carbon \
            --with-cholmod="-lcholmod -lmetis"
            LLVM_CONFIG=/opt/local/bin/llvm-config-mp-3.0

make


Ben