Re: MXE Octave: "... has no symbols" warning under Mac OS X

[Ben Abbott]

On Sep 20, 2013, at 12:21 PM, Anirudha Bose wrote:

> On Fri, Sep 20, 2013 at 9:03 PM, Ben Abbott <address@hidden> wrote:
> 
> On Sep 20, 2013, at 9:09 AM, Anirudha Bose wrote:
> 
> > On Wed, Sep 4, 2013 at 12:18 AM, Ben Abbott <address@hidden> wrote:
> >
> >> On Sep 3, 2013, at 1:40 PM, Anirudha Bose wrote:
> >>
> >> > On Tue, Sep 3, 2013 at 9:56 PM, Benjamin Abbott <address@hidden> wrote:
> >> >
> >> >> On Sep 3, 2013, at 11:57 AM, Anirudha Bose <address@hidden> wrote:
> >> >>
> >> >>> On Tue, Sep 3, 2013 at 6:18 PM, Ben Abbott <address@hidden> wrote:
> >> >>> On Sep 3, 2013, at 4:11 AM, Anirudha Bose wrote:
> >> >>>
> >> >>> >
> >> >>> > On Tue, Aug 27, 2013 at 7:58 PM, Alexander Hansen <address@hidden> 
> >> >>> > wrote:
> >> >>> >> On 8/27/13 4:58 AM, Ben Abbott wrote:
> >> >>> >>> On Aug 27, 2013, at 4:26 AM, Anirudha Bose wrote:
> >> >>> >>>
> >> >>> >>>> I have been working on reusing mxe-octave [1] to natively build 
> >> >>> >>>> Octave for Mac OS X. I have been able to complete the build 
> >> >>> >>>> process for the target i686-apple-darwin11, but the executable 
> >> >>> >>>> usr/bin/octave doesn't launch Octave. Here is the error I see.
> >> >>> >>>>
> >> >>> >>>> dyld: Library not loaded: libcholmod.so
> >> >>> >>>>    Referenced from: /Users/mac6-user1/./mxe-octave/usr/bin/octave
> >> >>> >>>>    Reason: image not found
> >> >>> >>>> Trace/BPT trap: 5
> >> >>> >>>>
> >> >>> >>>> I tried to trace back the error and saw that the library 
> >> >>> >>>> libcholmod.so belongs to the package suitesparse. The following 
> >> >>> >>>> is a part of the log of suitesparse,  which I think is 
> >> >>> >>>> responsible for the above error.
> >> >>> >>>>
> >> >>> >>>> ar rv  libcholmod.a cholmod_aat.o cholmod_add.o cholmod_band.o 
> >> >>> >>>> cholmod_change_factor.o cholmod_common.o cholmod_complex.o 
> >> >>> >>>> cholmod_copy.o cholmod_dens$
> >> >>> >>>> ar: creating archive libcholmod.a
> >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_ccolamd.o) has 
> >> >>> >>>> no symbols
> >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_csymamd.o) has 
> >> >>> >>>> no symbols
> >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_metis.o) has no 
> >> >>> >>>> symbols
> >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_nesdis.o) has 
> >> >>> >>>> no symbols
> >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_camd.o) has no 
> >> >>> >>>> symbols
> >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_l_ccolamd.o) 
> >> >>> >>>> has no symbols
> >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_l_csymamd.o) 
> >> >>> >>>> has no symbols
> >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_l_metis.o) has 
> >> >>> >>>> no symbols
> >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_l_nesdis.o) has 
> >> >>> >>>> no symbols
> >> >>> >>>> /opt/local/bin/ranlib: file: libcholmod.a(cholmod_l_camd.o) has 
> >> >>> >>>> no symbols
> >> >>> >>>>
> >> >>> >>>> I suppose the problem is because Mac OS "ar" is not standard. I 
> >> >>> >>>> have also noticed these warnings with other packages too. Can 
> >> >>> >>>> anyone tell me how to get a proper build under Mac OS X?
> >> >>> >>>>
> >> >>> >>>> [1] http://hg.octave.org/mxe-octave
> >> >>> >>>>
> >> >>> >>>> - Anirudha
> >> >>> >>>
> >> >>> >>> MacOSX's toolchain avoids the use of static libraries (I assume to 
> >> >>> >>> avoid violating GNU licenses).  That is likely at the root of the 
> >> >>> >>> problem.
> >> >>> >>>
> >> >>> >>> In any event, I expect that both Fink and Macports include an "ar" 
> >> >>> >>> with their basic installation.
> >> >>> >>>
> >> >>> >>> Ben
> >> >>> >>
> >> >>> >> You assume incorrectly in the case of Fink. :-)  We use the 
> >> >>> >> system's "ar".
> >> >>> >>
> >> >>> >> It looks to me like the problem could be that 
> >> >>> >> /Users/mac6-user1/./mxe-octave/usr/bin/octave wants to link 
> >> >>> >> "libcholmod.so", which would be 
> >> >>> >> /Users/mac6-user1/./mxe-octave/usr/bin/libcholmod.so .  You can 
> >> >>> >> check this scenario using "otool -L 
> >> >>> >> /Users/mac6-user1/./mxe-octave/usr/bin/octave" .
> >> >>> >
> >> >>> > This is exactly the reason why I am seeing the above errors. I am 
> >> >>> > using install_name_tool for changing the paths, but is there any way 
> >> >>> > to automate this job? The errors exist in too many executables and 
> >> >>> > shared libraries to be done manually one by one. I also recompiled 
> >> >>> > Octave with a new LDFLAG -headerpad_max_install_names to reserve 
> >> >>> > enough space for changing all dynamic shared library paths.
> >> >>>
> >> >>> Maybe there is a better way ... but I wrote scripts to handle this task
> >> >>>
> >> >>>         
> >> >>> http://wiki.octave.org/Create_a_MacOS_X_App_Bundle_Using_MacPorts#Fixing_.22dyld:_Library_not_loaded.22_Errors
> >> >>>
> >> >>> I got the following error when I executed your scripts.
> >> >>>
> >> >>> octave:1> dylibs_fix ("bin/octave", "lib", false)
> >> >>> Working in directory -> "lib" ...error: 'dylibs_isdylib' undefined 
> >> >>> near line 74 column 21
> >> >>> error: evaluating argument list element number 1
> >> >>> error: called from:
> >> >>> error:   
> >> >>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/dylibs_find.m
> >> >>>  at line 74, column 13
> >> >>> error:   
> >> >>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/dylibs_fix.m
> >> >>>  at line 160, column 12
> >> >>>
> >> >>> I have saved all the m-scripts at one place, and my working directory 
> >> >>> contained the bin and lib folders. Am I doing something wrong?
> >> >>>
> >> >>> - Anirudha
> >> >>
> >> >> Please add the directory containing the scripts to your path.
> >> >
> >> > This doesn't seem to solve my problem. After adding the directory to my 
> >> > path, this is what "echo $PATH" returns --
> >> >
> >> > /opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/local/upgraded/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/
> >> >
> >> > The last path contains all the 4 scripts. I opened Octave and executed 
> >> > dylibs_fix ("bin/octave", "lib", false), but got the same errors.
> >>
> >> The directory to the scripts needs to be added to Octave's path (not your 
> >> shell's path).
> >>
> >> Ben
> >
> > Hi Ben.
> >
> > I am in the process of creating Octave.app which will contain all the 
> > compiled binaries. I have been able to create an app bundle containing the 
> > "bin" and "lib" folders built with MXE. However the present problem is that 
> > the dynamic libraries refer to the libraries contained in the original path 
> > (in mxe directory), and not the libraries inside the app bundle (as 
> > reported by otool). I applied your scripts, but they did not work for me. 
> > Please see the log of your scripts [1] and the otool output [2].
> >
> > [1] http://pastebin.ca/2455382
> > [2] http://pastebin.ca/2455389
> >
> > Ideally, all the library path names should point to the relative paths 
> > inside the application bundle. Are your scripts capable of fixing  this, or 
> > I am doing something wrong? I am aware of some Python modules like py2app 
> > which solve such problems.
> >
> > -Anirudha
> 
> You need to point dylibs_fix to the actual Octave binary (not the directory 
> containing the binary). Notice the first problem reported is ...
> 
>         otool: can't map file: 
> /Users/mac6-user1/Documents/Octave-3.7.5.app/Contents/Resources/bin (Invalid 
> argument)
> 
> If the binary's file name is "octave-3.6.4", then I expect the command you 
> need is ...
> 
>         dylibs_fix("bin/octave-3.6.4","lib",true)
> 
>> After doing the above change, I am getting this result [1]. Note that aven 
>> after applying your scripts, I am getting the following results, which 
>> indicates that the MXE library paths are still there.
>> 
>>     IIITs-iMac-7:Resources mac6-user1$ otool -L ./lib/libreadline.6.2.dylib 
>> ./lib/libreadline.6.2.dylib:
>>     @executable_path/..//libreadline.6.2.dylib (compatibility version 6.0.0, 
>> current version 6.2.0)
>>     
>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/libncurses.5.dylib
>>  (compatibility version 5.0.0, current version 5.0.0)
>>     /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 
>> 169.3.0)

The current versions all look to be above the compatible versions, so I think 
you are ok there.  However, I'm not sure why they are not the same ... maybe 
this is normal, but it looks unexpected to me.  For now, I'd just assume this 
is normal behavior.

>> IIITs-iMac-7:Resources mac6-user1$ otool -L 
>> ./lib/octave/3.7.5/liboctinterp.1.dylib 
>> ./lib/octave/3.7.5/liboctinterp.1.dylib:
>>     @executable_path/..//liboctinterp.1.dylib (compatibility version 2.0.0, 
>> current version 2.1.0)
>>     /System/Library/Frameworks/JavaVM.framework/Versions/A/JavaVM 
>> (compatibility version 1.0.0, current version 1.0.0)
>>     
>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/octave/3.7.5/liboctave.1.dylib
>>  (compatibility version 2.0.0, current version 2.1.0)
>>     
>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/libfreetype.6.dylib
>>  (compatibility version 16.0.0, current version 16.0.0)
>>     
>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/libhdf5.7.dylib
>>  (compatibility version 8.0.0, current version 8.4.0)
>>     
>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/libz.1.dylib
>>  (compatibility version 1.0.0, current version 1.2.7)
>>     
>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/libfftw3_threads.3.dylib
>>  (compatibility version 7.0.0, current version 7.2.0)
>>     
>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/libfftw3.3.dylib
>>  (compatibility version 7.0.0, current version 7.2.0)
>>     
>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/libfftw3f_threads.3.dylib
>>  (compatibility version 7.0.0, current version 7.2.0)
>>     
>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/libfftw3f.3.dylib
>>  (compatibility version 7.0.0, current version 7.2.0)
>>     
>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/libpcre.1.dylib
>>  (compatibility version 2.0.0, current version 2.1.0)
>>     
>> /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/libfontconfig.1.dylib
>>  (compatibility version 9.0.0, current version 9.0.0)
>>     /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 
>> 169.3.0)
>>     /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL 
>> (compatibility version 1.0.0, current version 1.0.0)
>>     /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon 
>> (compatibility version 2.0.0, current version 155.0.0)
>>     /System/Library/Frameworks/vecLib.framework/Versions/A/vecLib 
>> (compatibility version 1.0.0, current version 380.6.0)
>>     /opt/local/lib/libgcc/libstdc++.6.dylib (compatibility version 7.0.0, 
>> current version 7.18.0)
>>     
>> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
>>  (compatibility version 150.0.0, current version 744.12.0)
>>     
>> /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
>>  (compatibility version 1.0.0, current version 45.0.0)
>> 
>> [1] http://pastebin.ca/2455538

The stuff under "/System/Library" should be ok, since this is part of MacOSX.  
As before, the current versions are greater than the compatible versions, so 
all looks ok.

> Also the warnings below need to be resolved.
> 
>         warning: dylibs_fix: Missing dynamic library "libgfortran.3.dylib" 
> needed by "libarpack.so"
>         warning: dylibs_fix: Missing dynamic library "libgcc_s.1.dylib" 
> needed by "libarpack.so"
>         warning: dylibs_fix: Missing dynamic library "libquadmath.0.dylib" 
> needed by "libarpack.so"
> 
> Where are these missing libraries located?  Did dylibs_fix miss them or are 
> they not yet in your bundle?
> 
>> I am using system GCC to build Octave for Mac OS X. So the above libraries 
>> are not in my build directory, but in the following paths:
>>     /opt/local/lib/gcc47/
>>     /opt/local/lib/libgcc/

The stuff in opt/local is not part of the MacOSX system, but is part of 
Macports.  We've probably been through this before, but ... Is there no way to 
use only MXE compilers and the MacOSX system libraries, which will be in 
/usr/lib?  If not, what is the advantage of using MXE and Macports, rather than 
just Macports?

Ben