Re: [PATCH] gnu: Add openldap.

[Ludovic Courtès]

Andreas Enge <address@hidden> skribis:

> Attached a patch for openldap, the test phase of which was really tricky.

Builds fine for me when tests are turned off.

> Since there are dynamically linked independent server and client programs, 
> I had to set LD_LIBRARY_PATH.

That’s a problem.  The problem is that the servers/slapd executable ends
up without libdb in its RUNPATH:

--8<---------------cut here---------------start------------->8---
$ objdump -x 
/tmp/nix-build-openldap-2.4.33.drv-0/openldap-2.4.33/tests/../servers/slapd/slapd
 |grep RUNPATH
  RUNPATH              
/nix/store/n0m40arwkv84ispnah1hma7526bw0llk-cyrus-sasl-2.1.26/lib:/nix/store/9fnjjsbarscbmakr44ixfv9yhg6z12mw-glibc-2.17/lib:/nix/store/lwc6sygaglzfk17v3w15cc2xv97fjgci-gcc-4.7.2/lib64:/nix/store/lwc6sygaglzfk17v3w15cc2xv97fjgci-gcc-4.7.2/lib:/nix/store/b8vagvbwasqwy8wv8545np40ra468c3p-icu4c-50.1.1/lib:/nix/store/ip943rzyiak1k06h7p2srily1wz73hsd-openssl-1.0.1c/lib
--8<---------------cut here---------------end--------------->8---

That binary gets built like this:

--8<---------------cut here---------------start------------->8---
/nix/store/ryk1ywzz31kp4biclxq3yq6hpjycalyy-bash-4.2/bin/sh ../../libtool 
--mode=link gcc -static -g -O2     -o slapd main.o globals.o bconfig.o config.o 
daemon.o connection.o search.o filter.o add.o cr.o attr.o entry.o backend.o 
backends.o result.o operation.o dn.o compare.o modify.o delete.o modrdn.o 
ch_malloc.o value.o ava.o bind.o unbind.o abandon.o filterentry.o phonetic.o 
acl.o str2filter.o aclparse.o init.o user.o lock.o controls.o extended.o 
passwd.o schema.o schema_check.o schema_init.o schema_prep.o schemaparse.o ad.o 
at.o mr.o syntax.o oc.o saslauthz.o oidm.o starttls.o index.o sets.o referral.o 
root_dse.o sasl.o module.o mra.o mods.o sl_malloc.o zn_malloc.o limits.o 
operational.o matchedValues.o cancel.o syncrepl.o backglue.o backover.o 
ctxcsn.o ldapsync.o frontend.o slapadd.o slapcat.o slapcommon.o slapdn.o 
slapindex.o slappasswd.o slaptest.o slapauth.o slapacl.o component.o aci.o 
alock.o txn.o slapschema.o  version.o libbackends.a liboverlays.a 
../../libraries/liblunicode/liblunicode.a 
../../libraries/librewrite/librewrite.a ../../libraries/liblutil/liblutil.a 
../../libraries/libldap_r/libldap_r.la ../../libraries/liblber/liblber.la  
-ldb-5.3      -licuuc -licudata -lsasl2  -lssl -lcrypto   -lresolv  -pthread  \
        
mkdir .libs
gcc -g -O2 -o slapd main.o globals.o bconfig.o config.o daemon.o connection.o 
search.o filter.o add.o cr.o attr.o entry.o backend.o backends.o result.o 
operation.o dn.o compare.o modify.o delete.o modrdn.o ch_malloc.o value.o ava.o 
bind.o unbind.o abandon.o filterentry.o phonetic.o acl.o str2filter.o 
aclparse.o init.o user.o lock.o controls.o extended.o passwd.o schema.o 
schema_check.o schema_init.o schema_prep.o schemaparse.o ad.o at.o mr.o 
syntax.o oc.o saslauthz.o oidm.o starttls.o index.o sets.o referral.o 
root_dse.o sasl.o module.o mra.o mods.o sl_malloc.o zn_malloc.o limits.o 
operational.o matchedValues.o cancel.o syncrepl.o backglue.o backover.o 
ctxcsn.o ldapsync.o frontend.o slapadd.o slapcat.o slapcommon.o slapdn.o 
slapindex.o slappasswd.o slaptest.o slapauth.o slapacl.o component.o aci.o 
alock.o txn.o slapschema.o version.o -pthread  libbackends.a liboverlays.a 
../../libraries/liblunicode/liblunicode.a 
../../libraries/librewrite/librewrite.a ../../libraries/liblutil/liblutil.a 
../../libraries/libldap_r/.libs/libldap_r.a 
/tmp/nix-build-openldap-2.4.33.drv-0/openldap-2.4.33/libraries/liblber/.libs/liblber.a
 ../../libraries/liblber/.libs/liblber.a 
/nix/store/c6hmllvdzgphyfb9pwaad42mzdf3kx8w-bdb-5.3.21/lib/libdb-5.3.so 
-lpthread -licuuc -licudata 
/nix/store/n0m40arwkv84ispnah1hma7526bw0llk-cyrus-sasl-2.1.26/lib/libsasl2.so 
-ldl -lssl -lcrypto -lresolv -pthread -Wl,--rpath 
-Wl,/nix/store/n0m40arwkv84ispnah1hma7526bw0llk-cyrus-sasl-2.1.26/lib 
-Wl,--rpath 
-Wl,/nix/store/n0m40arwkv84ispnah1hma7526bw0llk-cyrus-sasl-2.1.26/lib
--8<---------------cut here---------------end--------------->8---

As you can see, libtool replaced -ldb-5.3 by the corresponding absolute
file name.

However, our ld-wrapper only adds to the RUNPATH directories when they
contain libraries passed via -l.  We can fix that eventually, but that’s
not the problem here.

Here’s a hint:

--8<---------------cut here---------------start------------->8---
$ /tmp/nix-build-openldap-2.4.33.drv-0/openldap-2.4.33/libtool --version
ltmain.sh (GNU libtool) 1.5.22-OpenLDAP (1.1220.2.365 2005/12/18 22:14:06)
--8<---------------cut here---------------end--------------->8---

This is not only old, but also hacked in unknown ways.

So, here’s what I tried:

--8<---------------cut here---------------start------------->8---
address@hidden:/tmp/nix-build-openldap-2.4.33.drv-0/openldap-2.4.33]$ cp `type 
-P libtool` .

address@hidden:/tmp/nix-build-openldap-2.4.33.drv-0/openldap-2.4.33]$ ./libtool 
--version
libtool (GNU libtool) 2.4.2
Written by Gordon Matzigkeit <address@hidden>, 1996

Copyright (C) 2011 Free Software Foundation, Inc.  
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

address@hidden:/tmp/nix-build-openldap-2.4.33.drv-0/openldap-2.4.33]$ bash

address@hidden:/tmp/nix-build-openldap-2.4.33.drv-0/openldap-2.4.33]$ source 
../environment-variables
address@hidden:/tmp/nix-build-openldap-2.4.33.drv-0]$ rm servers/slapd/slapd

address@hidden:/tmp/nix-build-openldap-2.4.33.drv-0]$ make -C servers/slapd/

[...]

/nix/store/ryk1ywzz31kp4biclxq3yq6hpjycalyy-bash-4.2/bin/sh ../../libtool 
--mode=link gcc -static -g -O2     -o slapd main.o globals.o bconfig.o config.o 
daemon.o connection.o search.o filter.o add.o cr.o attr.o entry.o backend.o 
backends.o result.o operation.o dn.o compare.o modify.o delete.o modrdn.o 
ch_malloc.o value.o ava.o bind.o unbind.o abandon.o filterentry.o phonetic.o 
acl.o str2filter.o aclparse.o init.o user.o lock.o controls.o extended.o 
passwd.o schema.o schema_check.o schema_init.o schema_prep.o schemaparse.o ad.o 
at.o mr.o syntax.o oc.o saslauthz.o oidm.o starttls.o index.o sets.o referral.o 
root_dse.o sasl.o module.o mra.o mods.o sl_malloc.o zn_malloc.o limits.o 
operational.o matchedValues.o cancel.o syncrepl.o backglue.o backover.o 
ctxcsn.o ldapsync.o frontend.o slapadd.o slapcat.o slapcommon.o slapdn.o 
slapindex.o slappasswd.o slaptest.o slapauth.o slapacl.o component.o aci.o 
alock.o txn.o slapschema.o  version.o libbackends.a liboverlays.a 
../../libraries/liblunicode/liblunicode.a 
../../libraries/librewrite/librewrite.a ../../libraries/liblutil/liblutil.a 
../../libraries/libldap_r/libldap_r.la ../../libraries/liblber/liblber.la  
-ldb-5.3      -licuuc -licudata -lsasl2  -lssl -lcrypto   -lresolv  -pthread  \

libtool: link: gcc -g -O2 -o slapd main.o globals.o bconfig.o config.o daemon.o 
connection.o search.o filter.o add.o cr.o attr.o entry.o backend.o backends.o 
result.o operation.o dn.o compare.o modify.o delete.o modrdn.o ch_malloc.o 
value.o ava.o bind.o unbind.o abandon.o filterentry.o phonetic.o acl.o 
str2filter.o aclparse.o init.o user.o lock.o controls.o extended.o passwd.o 
schema.o schema_check.o schema_init.o schema_prep.o schemaparse.o ad.o at.o 
mr.o syntax.o oc.o saslauthz.o oidm.o starttls.o index.o sets.o referral.o 
root_dse.o sasl.o module.o mra.o mods.o sl_malloc.o zn_malloc.o limits.o 
operational.o matchedValues.o cancel.o syncrepl.o backglue.o backover.o 
ctxcsn.o ldapsync.o frontend.o slapadd.o slapcat.o slapcommon.o slapdn.o 
slapindex.o slappasswd.o slaptest.o slapauth.o slapacl.o component.o aci.o 
alock.o txn.o slapschema.o version.o -pthread  libbackends.a liboverlays.a 
../../libraries/liblunicode/liblunicode.a 
../../libraries/librewrite/librewrite.a ../../libraries/liblutil/liblutil.a 
../../libraries/libldap_r/.libs/libldap_r.a 
/tmp/nix-build-openldap-2.4.33.drv-0/openldap-2.4.33/libraries/liblber/.libs/liblber.a
 ../../libraries/liblber/.libs/liblber.a -ldb-5.3 -licuuc -licudata 
/nix/store/n0m40arwkv84ispnah1hma7526bw0llk-cyrus-sasl-2.1.26/lib/libsasl2.so 
-ldl -lssl -lcrypto -lresolv -pthread -Wl,-rpath 
-Wl,/nix/store/n0m40arwkv84ispnah1hma7526bw0llk-cyrus-sasl-2.1.26/lib 
-Wl,-rpath -Wl,/nix/store/n0m40arwkv84ispnah1hma7526bw0llk-cyrus-sasl-2.1.26/lib
rm -f slapadd slapcat slapdn slapindex slappasswd slaptest slapauth slapacl 
slapschema
--8<---------------cut here---------------end--------------->8---

Surprise, -ldb5.3 is used now, and thus it ends up in the RPATH.

So I would suggest adding Libtool/bin as an input, and adding a phase
that does (copy-file (which "libtool") "libtool") right after ‘configure’.

> Still, one test failed, but not all the time: It succeeded once when
> run in the build tree. So I took it out. Even then, some test fails
> from time to time; sometimes after an hour of different tests passing,
> sometimes after 15 minutes.

Did you check whether it does DNS lookups or similar?

If it turns out to be this unreliable, then yes, it’s safer to disable
them.

Thanks,
Ludo’.