Re: MXE Octave: "... has no symbols" warning under Mac OS X

[Ben Abbott]

On Sep 20, 2013, at 2:51 PM, Anirudha Bose wrote:

> On Fri, Sep 20, 2013 at 11:19 PM, Ben Abbott <address@hidden> wrote:
> 
>> On Sep 20, 2013, at 1:05 PM, Anirudha Bose wrote:
>> 
>> > On Fri, Sep 20, 2013 at 10:14 PM, Ben Abbott <address@hidden> wrote:
>> >
>> > On Sep 20, 2013, at 12:21 PM, Anirudha Bose wrote:
>> >
>> > > On Fri, Sep 20, 2013 at 9:03 PM, Ben Abbott <address@hidden> wrote:
>> > >
>> > > On Sep 20, 2013, at 9:09 AM, Anirudha Bose wrote:
>> > >
>> > > > On Wed, Sep 4, 2013 at 12:18 AM, Ben Abbott <address@hidden> wrote:
>> > > >
>> > > >> On Sep 3, 2013, at 1:40 PM, Anirudha Bose wrote:
>> > > >>
>> > > >> > On Tue, Sep 3, 2013 at 9:56 PM, Benjamin Abbott <address@hidden> 
>> > > >> > wrote:
>> > > >> >
>> > > >> >> On Sep 3, 2013, at 11:57 AM, Anirudha Bose <address@hidden> wrote:
>> > > >> >>
>> > > >> >>> On Tue, Sep 3, 2013 at 6:18 PM, Ben Abbott <address@hidden> wrote:
>> > > >> >>> On Sep 3, 2013, at 4:11 AM, Anirudha Bose wrote:
>> > > >> >>>
>> > > >> >>> >
>> > > >> >>> > On Tue, Aug 27, 2013 at 7:58 PM, Alexander Hansen 
>> > > >> >>> > <address@hidden> wrote:
>> > > >> >>> >> On 8/27/13 4:58 AM, Ben Abbott wrote:
>> > > >> >>> >>> On Aug 27, 2013, at 4:26 AM, Anirudha Bose wrote:
>> > > >> >>> >>>
>> > > >> >>> >>>> I have been working on reusing mxe-octave [1] to natively 
>> > > >> >>> >>>> build Octave for Mac OS X. I have been able to complete the 
>> > > >> >>> >>>> build process for the target i686-apple-darwin11, but the 
>> > > >> >>> >>>> executable usr/bin/octave doesn't launch Octave. Here is the 
>> > > >> >>> >>>> error I see.
>> > > >> >>> >>>>
>> > > >> >>> >>>> dyld: Library not loaded: libcholmod.so
>> > > >> >>> >>>>    Referenced from: 
>> > > >> >>> >>>> /Users/mac6-user1/./mxe-octave/usr/bin/octave
>> > > >> >>> >>>>    Reason: image not found
>> > > >> >>> >>>> Trace/BPT trap: 5
>> > > >> >>> >>>>
>> > > >> >>> >>>> I tried to trace back the error and saw that the library 
>> > > >> >>> >>>> libcholmod.so belongs to the package suitesparse. The 
>> > > >> >>> >>>> following is a part of the log of suitesparse,  which I 
>> > > >> >>> >>>> think is responsible for the above error.
>> > > >> >>> >>>>
>> > > >> >>> >>>> ar rv  libcholmod.a cholmod_aat.o cholmod_add.o 
>> > > >> >>> >>>> cholmod_band.o cholmod_change_factor.o cholmod_common.o 
>> > > >> >>> >>>> cholmod_complex.o cholmod_copy.o cholmod_dens$
>> > > >> >>> >>>> ar: creating archive libcholmod.a
>> > > >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_ccolamd.o) 
>> > > >> >>> >>>> has no symbols
>> > > >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_csymamd.o) 
>> > > >> >>> >>>> has no symbols
>> > > >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_metis.o) 
>> > > >> >>> >>>> has no symbols
>> > > >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_nesdis.o) 
>> > > >> >>> >>>> has no symbols
>> > > >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_camd.o) 
>> > > >> >>> >>>> has no symbols
>> > > >> >>> >>>> /opt/local/bin/ranlib: file: 
>> > > >> >>> >>>> libcholmod.a(cholmod_l_ccolamd.o) has no symbols
>> > > >> >>> >>>> /opt/local/bin/ranlib: file: 
>> > > >> >>> >>>> libcholmod.a(cholmod_l_csymamd.o) has no symbols
>> > > >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_l_metis.o) 
>> > > >> >>> >>>> has no symbols
>> > > >> >>> >>>> /opt/local/bin/ranlib: file: 
>> > > >> >>> >>>> libcholmod.a(cholmod_l_nesdis.o) has no symbols
>> > > >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_l_camd.o) 
>> > > >> >>> >>>> has no symbols
>> > > >> >>> >>>>
>> > > >> >>> >>>> I suppose the problem is because Mac OS "ar" is not 
>> > > >> >>> >>>> standard. I have also noticed these warnings with other 
>> > > >> >>> >>>> packages too. Can anyone tell me how to get a proper build 
>> > > >> >>> >>>> under Mac OS X?
>> > > >> >>> >>>>
>> > > >> >>> >>>> [1] http://hg.octave.org/mxe-octave
>> > > >> >>> >>>>
>> > > >> >>> >>>> - Anirudha
>> > > >> >>> >>>
>> > > >> >>> >>> MacOSX's toolchain avoids the use of static libraries (I 
>> > > >> >>> >>> assume to avoid violating GNU licenses).  That is likely at 
>> > > >> >>> >>> the root of the problem.
>> > > >> >>> >>>
>> > > >> >>> >>> In any event, I expect that both Fink and Macports include an 
>> > > >> >>> >>> "ar" with their basic installation.
>> > > >> >>> >>>
>> > > >> >>> >>> Ben
>> > > >> >>> >>
>> > > >> >>> >> You assume incorrectly in the case of Fink. :-)  We use the 
>> > > >> >>> >> system's "ar".
>> > > >> >>> >>
>> > > >> >>> >> It looks to me like the problem could be that 
>> > > >> >>> >> /Users/mac6-user1/./mxe-octave/usr/bin/octave wants to link 
>> > > >> >>> >> "libcholmod.so", which would be 
>> > > >> >>> >> /Users/mac6-user1/./mxe-octave/usr/bin/libcholmod.so .  You 
>> > > >> >>> >> can check this scenario using "otool -L 
>> > > >> >>> >> /Users/mac6-user1/./mxe-octave/usr/bin/octave" .
>> > > >> >>> >
>> > > >> >>> > This is exactly the reason why I am seeing the above errors. I 
>> > > >> >>> > am using install_name_tool for changing the paths, but is there 
>> > > >> >>> > any way to automate this job? The errors exist in too many 
>> > > >> >>> > executables and shared libraries to be done manually one by 
>> > > >> >>> > one. I also recompiled Octave with a new LDFLAG 
>> > > >> >>> > -headerpad_max_install_names to reserve enough space for 
>> > > >> >>> > changing all dynamic shared library paths.
>> > > >> >>>
>> > > >> >>> Maybe there is a better way ... but I wrote scripts to handle 
>> > > >> >>> this task
>> > > >> >>>
>> > > >> >>>        
>> > > >> >>> http://wiki.octave.org/Create_a_MacOS_X_App_Bundle_Using_MacPorts#Fixing_.22dyld:_Library_not_loaded.22_Errors
>> > > >> >>>
>> > > >> >>> I got the following error when I executed your scripts.
>> > > >> >>>
>> > > >> >>> octave:1> dylibs_fix ("bin/octave", "lib", false)
>> > > >> >>> Working in directory -> "lib" ...error: 'dylibs_isdylib' 
>> > > >> >>> undefined near line 74 column 21
>> > > >> >>> error: evaluating argument list element number 1
>> > > >> >>> error: called from:
>> > > >> >>> error:   
>> > > >> >>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/dylibs_find.m
>> > > >> >>>  at line 74, column 13
>> > > >> >>> error:   
>> > > >> >>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/dylibs_fix.m
>> > > >> >>>  at line 160, column 12
>> > > >> >>>
>> > > >> >>> I have saved all the m-scripts at one place, and my working 
>> > > >> >>> directory contained the bin and lib folders. Am I doing something 
>> > > >> >>> wrong?
>> > > >> >>>
>> > > >> >>> - Anirudha
>> > > >> >>
>> > > >> >> Please add the directory containing the scripts to your path.
>> > > >> >
>> > > >> > This doesn't seem to solve my problem. After adding the directory 
>> > > >> > to my path, this is what "echo $PATH" returns --
>> > > >> >
>> > > >> > /opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/local/upgraded/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/
>> > > >> >
>> > > >> > The last path contains all the 4 scripts. I opened Octave and 
>> > > >> > executed dylibs_fix ("bin/octave", "lib", false), but got the same 
>> > > >> > errors.
>> > > >>
>> > > >> The directory to the scripts needs to be added to Octave's path (not 
>> > > >> your shell's path).
>> > > >>
>> > > >> Ben
>> > > >
>> > > > Hi Ben.
>> > > >
>> > > > I am in the process of creating Octave.app which will contain all the 
>> > > > compiled binaries. I have been able to create an app bundle containing 
>> > > > the "bin" and "lib" folders built with MXE. However the present 
>> > > > problem is that the dynamic libraries refer to the libraries contained 
>> > > > in the original path (in mxe directory), and not the libraries inside 
>> > > > the app bundle (as reported by otool). I applied your scripts, but 
>> > > > they did not work for me. Please see the log of your scripts [1] and 
>> > > > the otool output [2].
>> > > >
>> > > > [1] http://pastebin.ca/2455382
>> > > > [2] http://pastebin.ca/2455389
>> > > >
>> > > > Ideally, all the library path names should point to the relative paths 
>> > > > inside the application bundle. Are your scripts capable of fixing  
>> > > > this, or I am doing something wrong? I am aware of some Python modules 
>> > > > like py2app which solve such problems.
>> > > >
>> > > > -Anirudha
>> > >
>> > > You need to point dylibs_fix to the actual Octave binary (not the 
>> > > directory containing the binary). Notice the first problem reported is 
>> > > ...
>> > >
>> > >         otool: can't map file: 
>> > > /Users/mac6-user1/Documents/Octave-3.7.5.app/Contents/Resources/bin 
>> > > (Invalid argument)
>> > >
>> > > If the binary's file name is "octave-3.6.4", then I expect the command 
>> > > you need is ...
>> > >
>> > >         dylibs_fix("bin/octave-3.6.4","lib",true)
>> > >
>> > >> After doing the above change, I am getting this result [1]. Note that 
>> > >> aven after applying your scripts, I am getting the following results, 
>> > >> which indicates that the MXE library paths are still there.
>> > >>
>> > >>     IIITs-iMac-7:Resources mac6-user1$ otool -L 
>> > >> ./lib/libreadline.6.2.dylib
>> > >> ./lib/libreadline.6.2.dylib:
>> > >>     @executable_path/..//libreadline.6.2.dylib (compatibility version 
>> > >> 6.0.0, current version 6.2.0)
>> > >>     
>> > >> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/libncurses.5.dylib
>> > >>  (compatibility version 5.0.0, current version 5.0.0)
>> > >>     /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
>> > >> version 169.3.0)
>> >
>> > The current versions all look to be above the compatible versions, so I 
>> > think you are ok there.  However, I'm not sure why they are not the same 
>> > ... maybe this is normal, but it looks unexpected to me.  For now, I'd 
>> > just assume this is normal behavior.
>> >
>> > >> IIITs-iMac-7:Resources mac6-user1$ otool -L 
>> > >> ./lib/octave/3.7.5/liboctinterp.1.dylib
>> > >> ./lib/octave/3.7.5/liboctinterp.1.dylib:
>> > >>     @executable_path/..//liboctinterp.1.dylib (compatibility version 
>> > >> 2.0.0, current version 2.1.0)
>> > >>     /System/Library/Frameworks/JavaVM.framework/Versions/A/JavaVM 
>> > >> (compatibility version 1.0.0, current version 1.0.0)
>> > >>     
>> > >> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/octave/3.7.5/liboctave.1.dylib
>> > >>  (compatibility version 2.0.0, current version 2.1.0)
>> >
>> > Don't you think the above path should be relative? It points to the 
>> > library inside MXE directory.
>> 
>> ah ... ok. From the last Macports bundle I attempted I see ...
>> 
>> $ otool -L liboctinterp.dylib
>> liboctinterp.dylib:
>>         @executable_path/../lib/octave/3.7.0+/liboctinterp.1.dylib 
>> (compatibility version 2.0.0, current version 2.1.0)
>>         @executable_path/../lib/octave/3.7.0+/liboctave.1.dylib 
>> (compatibility version 2.0.0, current version 2.1.0)
>>         @executable_path/../lib/octave/3.7.0+/libcruft.1.dylib 
>> (compatibility version 2.0.0, current version 2.0.0)
>>         @executable_path/../lib/libarpack.2.dylib (compatibility version 
>> 3.0.0, current version 3.0.0)
>>         @executable_path/../lib/libtatlas.dylib (compatibility version 
>> 3.10.0, current version 3.10.0)
>>         @executable_path/../lib/libqrupdate.1.dylib (compatibility version 
>> 0.0.0, current version 0.0.0)
>>         @executable_path/../lib/libreadline.6.2.dylib (compatibility version 
>> 6.0.0, current version 6.2.0)
>>         @executable_path/../lib/libncurses.5.dylib (compatibility version 
>> 5.0.0, current version 5.0.0)
>>         /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
>> version 159.1.0)
>>         @executable_path/../lib/gcc45/libgfortran.3.dylib (compatibility 
>> version 4.0.0, current version 4.0.0)
>>         @executable_path/../lib/libfltk_gl.1.3.dylib (compatibility version 
>> 1.3.0, current version 1.3.0)
>>         /System/Library/Frameworks/AGL.framework/Versions/A/AGL 
>> (compatibility version 1.0.0, current version 1.0.0)
>>         /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL 
>> (compatibility version 1.0.0, current version 1.0.0)
>>         
>> /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
>>  (compatibility version 1.0.0, current version 41.0.0)
>>         @executable_path/../lib/libfltk.1.3.dylib (compatibility version 
>> 1.3.0, current version 1.3.0)
>>         /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa 
>> (compatibility version 1.0.0, current version 17.0.0)
>>         @executable_path/../lib/libhdf5.7.dylib (compatibility version 
>> 8.0.0, current version 8.3.0)
>>         @executable_path/../lib/libfftw3.3.dylib (compatibility version 
>> 7.0.0, current version 7.2.0)
>>         @executable_path/../lib/libfftw3f.3.dylib (compatibility version 
>> 7.0.0, current version 7.2.0)
>>         @executable_path/../lib/libpcre.1.dylib (compatibility version 
>> 2.0.0, current version 2.0.0)
>>         @executable_path/../lib/libfontconfig.1.dylib (compatibility version 
>> 7.0.0, current version 7.0.0)
>>         @executable_path/../lib/libiconv.2.dylib (compatibility version 
>> 8.0.0, current version 8.1.0)
>>         @executable_path/../lib/libfreetype.6.dylib (compatibility version 
>> 15.0.0, current version 15.1.0)
>>         @executable_path/../lib/libz.1.dylib (compatibility version 1.0.0, 
>> current version 1.2.7)
>>         @executable_path/../lib/libbz2.1.0.dylib (compatibility version 
>> 1.0.0, current version 1.0.6)
>>         @executable_path/../lib/libexpat.1.dylib (compatibility version 
>> 8.0.0, current version 8.0.0)
>>         /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon 
>> (compatibility version 2.0.0, current version 153.0.0)
>>         @executable_path/../lib/gcc45/libstdc++.6.dylib (compatibility 
>> version 7.0.0, current version 7.14.0)
>>         /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current 
>> version 1105.0.0)
>>         @executable_path/../lib/gcc45/libgcc_s.1.dylib (compatibility 
>> version 1.0.0, current version 1.0.0)
>>         
>> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
>>  (compatibility version 150.0.0, current version 635.21.0)
>> 
>> When you ran fix_dylibs() was you pwd() equal to 
>> "/Users/mac6-user1/Documents/Octave-3.7.5.app/Contents/Resources"?  If not, 
>> perhaps you applied install_name_tool to the MXE libraries?
> 
> My pwd is  /Users/mac6-user1/Documents/Octave-3.7.5.app/Contents/Resources 
> and all the four m-scripts are in that location. I executed the command: 
> dylibs_fix("bin/octave-3.7.5","lib",false) and here is the log: 
> http://pastebin.ca/2455616
> 
> If you see the log, you will find commands like --
> 
> install_name_tool -change 
> '@executable_path/../lib/octave/3.7.5/liboctave.1.dylib' 
> '@executable_path/../lib/octave/3.7.5/liboctave.1.dylib' 
> '/Users/mac6-user1/Documents/Octave-3.7.5.app/Contents/Resources/lib/octave/3.7.5/liboctave.1.dylib'

The syntax for "-change" is ...

        install_name_tool -change old new file

In my last attempt, I see ...

install_name_tool -change '/opt/local/lib/octave/3.7.0+/liboctave.1.dylib' 
'@executable_path/../lib/octave/3.7.0+/liboctave.1.dylib' 
'/Users/bpabbott/Development/macports/Standalone_Bundles/Octave/2012.07.28/Octave-X86_64-3.7.0+v12b/Octave-3.7.0+.app/Contents/Resources/lib/octave/3.7.0+/liboctave.1.dylib'

The script dylibs_fix.m function be modified to avoid calling install_name_tool 
in the "old" and "new" paths are identical, but it looks to me as if both your 
result and mine are consistent and are assigning relative paths.

> and 
> 
> install_name_tool -change '@executable_path/../lib/libklu.so' 
> '@executable_path/../lib/libklu.so' 
> '/Users/mac6-user1/Documents/Octave-3.7.5.app/Contents/Resources/lib/libklu.so'
> 

> I think the above commands do not change anything, and the correct 
> install_name_tool would be something like
> 
> install_name_tool -change '...the MXE path...' '...relative path in 
> bundle...' 
> '/Users/mac6-user1/Documents/Octave-3.7.5.app/Contents/Resources/lib/libklu.so'

The symbol @executable_path represents the relative path (it is the path to the 
octave binary). The old path for your MXE case appears to already be relative 
... maybe you ran the scripts on the files more than once? ... or perhaps

In any event, we should verify we are both using the same versions of the 
dylibs_*.m functions.  I've attached copies.  Please try these and let me know 
if ...

        otool -L ./lib/octave/3.7.5/liboctinterp.1.dylib

... then indicates a relative path to liboctave ...

        @executable_path/../lib/octave/3.7.5/liboctave.1.dylib

... or still uses the absolute path to the MXE version ...

        
/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/octave/3.7.5/liboctave.1.dylib

Ben

dylibs_find.m
Description: Binary data

dylibs_fix.m
Description: Binary data

dylibs_get_deps.m
Description: Binary data

dylibs_isdylib.m
Description: Binary data