Re: Concurrent extraction of convenience-library components on OS X

[Ralf Wildenhues]

[ dropping bug-libtool ]

Hi Peter,

* Peter O'Gorman wrote on Thu, May 28, 2009 at 01:32:14AM CEST:
> Ralf Wildenhues wrote:
> > Anyway; what about this patch?  Please note that the patch requires that
> > the second argument to func_extract_an_archive has an absolute path; not
> > sure whether that is universally true.

> I wonder if the archive for the test could be a little larger. When I
> applied the test suite change without the other changes, I found that it
> passed 2 of the 4 times I ran the test, ar completes unpacking that
> small archive pretty fast, it would take a bit longer if the archive
> were significantly larger.

On the darwin system I tested on, the test already failed every time, so
I figured that was sufficient.  But you have a point of course.

Can you try this patch and play with the two lists (the number of
objects and the number of link tries) until they are large enough to
give sufficient failure chance?  I will apply it then (only the
testsuite addition has been changed).

Thanks,
Ralf

    Fix concurrent extraction of convenience libraries on Darwin.
    
    * libltdl/m4/libtool.m4 (_LT_CMD_OLD_ARCHIVE): New libtool
    variable `lock_old_archive_extraction', set to `yes' on darwin.
    * doc/libtool.texi (libtool script contents): Document it.
    * libltdl/config/ltmain.m4sh (func_extract_an_archive): Lock
    `ar x' invocation if `lock_old_archive_extraction' is yes.
    * tests/darwin.at (darwin concurrent library extraction): New
    test.
    * NEWS: Update.
    Report by Akim Demaille.

diff --git a/NEWS b/NEWS
index 3656acd..5a57891 100644
--- a/NEWS
+++ b/NEWS
@@ -38,6 +38,9 @@ New in 2.2.8 2009-??-??: git version 2.2.7a, Libtool team:
     The regression affected mainly (arguably broken) cross compiles.
   - Fix long standing bug that caused compiler checks for Fortran and
     C++ compilers to run twice.
+  - Link mode works around a parallel build failure on Darwin 9.6.0 due
+    to the `ar' `flock'ing an archive upon extraction, by protecting the
+    extraction of convenience archives with a lock.
 
 * Miscellaneous changes:
 
diff --git a/doc/libtool.texi b/doc/libtool.texi
index 5e25531..a7872c6 100644
--- a/doc/libtool.texi
+++ b/doc/libtool.texi
@@ -5670,6 +5670,11 @@ these commands, libtool will proceed to link against 
@var{$objdir/$newlib}
 instead of @var{soname}.
 @end defvar
 
address@hidden lock_old_archive_extraction
+Set to @samp{yes} if the extraction of a static library requires locking
+the library file.  This is required on Darwin.
address@hidden defvar
+
 @defvar build
 @defvarx build_alias
 @defvarx build_os
diff --git a/libltdl/config/ltmain.m4sh b/libltdl/config/ltmain.m4sh
index 5609304..9bc947f 100644
--- a/libltdl/config/ltmain.m4sh
+++ b/libltdl/config/ltmain.m4sh
@@ -2222,7 +2222,18 @@ func_extract_an_archive ()
     $opt_debug
     f_ex_an_ar_dir="$1"; shift
     f_ex_an_ar_oldlib="$1"
-    func_show_eval "(cd \$f_ex_an_ar_dir && $AR x \"\$f_ex_an_ar_oldlib\")" 
'exit $?'
+    if test "$lock_old_archive_extraction" = yes; then
+      lockfile=$f_ex_an_ar_oldlib.lock
+      until $opt_dry_run || ln "$progpath" "$lockfile" 2>/dev/null; do
+       func_echo "Waiting for $lockfile to be removed"
+       sleep 2
+      done
+    fi
+    func_show_eval "(cd \$f_ex_an_ar_dir && $AR x \"\$f_ex_an_ar_oldlib\")" \
+                  'stat=$?; rm -f "$lockfile"; exit $stat'
+    if test "$lock_old_archive_extraction" = yes; then
+      $opt_dry_run || rm -f "$lockfile"
+    fi
     if ($AR t "$f_ex_an_ar_oldlib" | sort | sort -uc >/dev/null 2>&1); then
      :
     else
diff --git a/libltdl/m4/libtool.m4 b/libltdl/m4/libtool.m4
index 0670980..5f24fab 100644
--- a/libltdl/m4/libtool.m4
+++ b/libltdl/m4/libtool.m4
@@ -1312,10 +1312,19 @@ if test -n "$RANLIB"; then
   esac
   old_archive_cmds="$old_archive_cmds~\$RANLIB \$oldlib"
 fi
+
+case $host_os in
+  darwin*)
+    lock_old_archive_extraction=yes ;;
+  *)
+    lock_old_archive_extraction=no ;;
+esac
 _LT_DECL([], [old_postinstall_cmds], [2])
 _LT_DECL([], [old_postuninstall_cmds], [2])
 _LT_TAGDECL([], [old_archive_cmds], [2],
     [Commands used to build an old-style archive])
+_LT_DECL([], [lock_old_archive_extraction], [0],
+    [Whether to use a lock for old archive extraction])
 ])# _LT_CMD_OLD_ARCHIVE
 
 
diff --git a/tests/darwin.at b/tests/darwin.at
index 6284116..6b49eb2 100644
--- a/tests/darwin.at
+++ b/tests/darwin.at
@@ -1,6 +1,6 @@
 # darwin.at - tests specific to Mac OS X
 #
-#   Copyright (C) 2008 Free Software Foundation, Inc.
+#   Copyright (C) 2008, 2009 Free Software Foundation, Inc.
 #   Written by Peter O'Gorman, 2008
 #
 #   This file is part of GNU Libtool.
@@ -98,3 +98,53 @@ AT_CHECK([$LIBTOOL --mode=link --tag=CC $CC  -o main$EXEEXT 
$CPPFLAGS $CFLAGS $L
 
 PATH=$save_PATH
 AT_CLEANUP
+
+
+AT_SETUP([darwin concurrent library extraction])
+
+AT_DATA([foo.c], [[
+int foo (void) { return 0; }
+]])
+
+AT_DATA([bar.c], [[
+extern int foo1 (void);
+int bar (void) { return foo1 (); }
+]])
+cp bar.c baz.c
+
+objects=
+for obj in 1 2 3 4 5; do
+  sed "s/foo/foo$obj/" < foo.c > foo$obj.c
+  AT_CHECK([$LIBTOOL --mode=compile $CC $CPPFLAGS $CFLAGS -c foo$obj.c],
+          [], [ignore], [ignore])
+  objects="$objects foo$obj.lo"
+done
+AT_CHECK([$LIBTOOL --mode=compile $CC $CPPFLAGS $CFLAGS -c bar.c],
+        [], [ignore], [ignore])
+AT_CHECK([$LIBTOOL --mode=compile $CC $CPPFLAGS $CFLAGS -c baz.c],
+        [], [ignore], [ignore])
+AT_CHECK([$LIBTOOL --mode=link $CC $CFLAGS $LDFLAGS -o libfoo.la $objects],
+        [], [ignore], [ignore])
+
+# Hypothesis: concurrent convenience archive extraction works.
+for i in 1 2 3 4 5; do
+  rm -f libbar.la libbaz.la
+  AT_CHECK([($LIBTOOL --mode=link $CC $CFLAGS $LDFLAGS ]dnl
+          [  -o libbar.la bar.lo -rpath /foo libfoo.la) & ]dnl
+          [($LIBTOOL --mode=link $CC $CFLAGS $LDFLAGS ]dnl
+          [  -o libbaz.la baz.lo -rpath /foo libfoo.la) & ]dnl
+          [wait; test -f libbar.la && test -f libbaz.la],
+          [], [ignore], [ignore])
+done
+
+# Hypothesis: the lock is not used in dry run mode.
+eval "`$LIBTOOL --config | $EGREP '^(objdir)='`"
+# Next line is internal detail.
+lockfile=$objdir/libfoo.a.lock
+echo stamp > $lockfile
+AT_CHECK([$LIBTOOL --dry-run --mode=link $CC $CFLAGS $LDFLAGS ]dnl
+        [ -o libbar.la bar.lo -rpath /foo libfoo.la],
+        [], [ignore], [ignore])
+AT_CHECK([grep stamp $lockfile], [], [ignore])
+
+AT_CLEANUP