[Automake-commit] [SCM] GNU Automake branch, ng/master, updated. v1.12.1-604-g7adf67b

[Stefano Lattarini]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU Automake".

http://git.sv.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=7adf67ba872ce5fa85a32acbdcf187ab729942bc

The branch, ng/master has been updated
       via  7adf67ba872ce5fa85a32acbdcf187ab729942bc (commit)
       via  3ec230adffaec4c7ce7203b4ef83a9823406bac7 (commit)
       via  5138db4e2268f69e24fe70e447079c76b3f09940 (commit)
       via  9c00be325193b90d371bed01112e89bfc33a24f5 (commit)
       via  e8b7bb4f9f65a4b7df7410e2312c925dd324c79b (commit)
       via  327d58514029cb5ae5add15335d709ccb831634e (commit)
       via  a8a46a77d412b3bd5d371f5f35281b91efe1dc18 (commit)
      from  316e64d9acad05b566e60976e7ab971efa4038d0 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
commit 7adf67ba872ce5fa85a32acbdcf187ab729942bc
Merge: 316e64d 3ec230a
Author: Stefano Lattarini <address@hidden>
Date:   Mon Jul 9 16:22:00 2012 +0200

    Merge branch 'ng/clean-many' into ng/master
    
    * ng/clean-many:
      [ng] clean: do not exceed command line length limits, even with many files
      [ng] $(am__newline): will not be stripped unexpectedly
      [ng] coverage: $(am__newline) should never be stripped off
      [ng] am.xargs-map: be usable with huge lists
      [ng] am.xargs-map: new internal make function
      [ng] coverage: cleaning rules with lots of files to clean

commit 3ec230adffaec4c7ce7203b4ef83a9823406bac7
Author: Stefano Lattarini <address@hidden>
Date:   Sat Jul 7 21:11:00 2012 +0200

    [ng] clean: do not exceed command line length limits, even with many files
    
    * lib/am/clean.am (.am.rm-f, .am.rm-rf): New internal functions.
    (.am.clean-cmd.f, .am.clean-cmd.d): Rewritten using them and the
    'am.xargs-map' function, in a way that should prevent failures
    due to excessive command line lengths.
    
    Signed-off-by: Stefano Lattarini <address@hidden>

commit 5138db4e2268f69e24fe70e447079c76b3f09940
Author: Stefano Lattarini <address@hidden>
Date:   Sat Jul 7 20:45:52 2012 +0200

    [ng] $(am__newline): will not be stripped unexpectedly
    
    Not even if placed at the end of another macro's expansion.
    
    * lib/am/header-vars.am (am__newline): Improve definition, and
    update comments.
    * t/internals.sh: Update: some previously xfailing checks are
    now passing.
    
    Signed-off-by: Stefano Lattarini <address@hidden>

commit 9c00be325193b90d371bed01112e89bfc33a24f5
Author: Stefano Lattarini <address@hidden>
Date:   Sat Jul 7 20:38:03 2012 +0200

    [ng] coverage: $(am__newline) should never be stripped off
    
    But currently it is, if placed at the end of another macro expansion.
    The issue will be fixed by a later patch.  For the moment ...
    
    * t/internals.sh: ... expose it here, with two new xfailing checks.
    
    Signed-off-by: Stefano Lattarini <address@hidden>

commit e8b7bb4f9f65a4b7df7410e2312c925dd324c79b
Author: Stefano Lattarini <address@hidden>
Date:   Sat Jul 7 19:09:32 2012 +0200

    [ng] am.xargs-map: be usable with huge lists
    
    The previous implementation could eat all memory if the given list was
    really huge -- but this function will be needed to operate on potentially
    huge lists! (i.e., list of tests or files which can exceed the command-line
    length limit of the system).
    
    As a drawback, the function cannot recursively invoke itself anymore (not
    even indirectly), but this is an acceptable limitation for its intended
    uses.
    
    * lib/am/header-vars.am (am.xargs-map): Reimplemented.
    (am.max-cmdline-args): Redefine, as its semantics have changed.
    (am.max-cmdline-args+1): Remove, no more needed.
    
    Signed-off-by: Stefano Lattarini <address@hidden>

commit 327d58514029cb5ae5add15335d709ccb831634e
Author: Stefano Lattarini <address@hidden>
Date:   Mon Jul 2 21:35:49 2012 +0200

    [ng] am.xargs-map: new internal make function
    
    It will be useful in future changes, to help us to fend of errors
    about "exceeded command line length" in our recipes.
    
    * lib/am/header-vars.am (am.xargs-map): New internal function.
    (am.max-cmdline-args, am.max-cmdline-args+1): New internal
    auxiliary make variables.
    * t/am-xargs-map.sh: New test.
    
    Signed-off-by: Stefano Lattarini <address@hidden>

commit a8a46a77d412b3bd5d371f5f35281b91efe1dc18
Author: Stefano Lattarini <address@hidden>
Date:   Mon Jul 2 12:04:36 2012 +0200

    [ng] coverage: cleaning rules with lots of files to clean
    
    * t/clean-many.sh: New test, try cleaning ~ 130 thousands files.
    Currently fails on some systems (e.g., NetBSD 5.1).
    * t/clean-many2.sh: New test, check that our cleaning rules do not
    hit errors due to an exceeded command line length when there are
    many files to clean; it does so by faking  a very low command line
    length limit for 'rm' (max 50 arguments).  Currently failing.
    
    Signed-off-by: Stefano Lattarini <address@hidden>

-----------------------------------------------------------------------

Summary of changes:
 lib/am/clean.am       |    7 ++-
 lib/am/header-vars.am |   40 ++++++++++++++-
 t/am-xargs-map.sh     |  139 +++++++++++++++++++++++++++++++++++++++++++++++++
 t/clean-many.sh       |   63 ++++++++++++++++++++++
 t/clean-many2.sh      |   81 ++++++++++++++++++++++++++++
 t/internals.tap       |   33 +++++++++---
 6 files changed, 352 insertions(+), 11 deletions(-)
 create mode 100755 t/am-xargs-map.sh
 create mode 100755 t/clean-many.sh
 create mode 100755 t/clean-many2.sh

diff --git a/lib/am/clean.am b/lib/am/clean.am
index d05cd66..bc11bc4 100644
--- a/lib/am/clean.am
+++ b/lib/am/clean.am
@@ -14,8 +14,11 @@
 ## You should have received a copy of the GNU General Public License
 ## along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-.am.clean-cmd.f = $(if $(strip $1),rm -f $(strip $1))
-.am.clean-cmd.d = $(if $(strip $1),rm -rf $(strip $1))
+.am.rm-f = $(if $(strip $1),rm -f $(strip $1)$(am__newline))
+.am.rm-rf = $(if $(strip $1),rm -rf $(strip $1)$(am__newline))
+
+.am.clean-cmd.f = $(call am.xargs-map,.am.rm-f,$1)
+.am.clean-cmd.d = $(call am.xargs-map,.am.rm-rf,$1)
 
 am__mostlyclean_files += $(MOSTLYCLEANFILES)
 am__clean_files       += $(CLEANFILES)
diff --git a/lib/am/header-vars.am b/lib/am/header-vars.am
index 26716f4..4ba656e 100644
--- a/lib/am/header-vars.am
+++ b/lib/am/header-vars.am
@@ -50,9 +50,11 @@ am__dquote := "
 am__squote := '
 # "` # Fix font-lock.
 
+# A literal newline character, that does not get stripped if used
+# at the end of the expansion of another macro.
 define am__newline
 
-
+$(am__empty)
 endef
 
 ## Makefiles generated by Automake-NG require GNU make >= 3.81.
@@ -256,6 +258,42 @@ am__strip_suffixes = $(strip \
           $(patsubst %$(firstword $1),%$(am__private_suffix),$2))), \
     $2))
 
+# Helper variables and function to help in recipes that could exceed
+# the command line length limit.
+
+## FIXME: Forty aguments; this is basically arbitrary.  In the long term,
+## FIXME: defining this after a configure-time test on the command-line
+## FIXME: length limits, or at least on a system-by-system basis, might
+## FIXME: be better.  But don't make it too big, or our implementation
+## FIXME: will likely suffer in performance and memory consumption.
+# And in the Information Age, we somehow managed to revert to abacus-like
+# counting.  Yay for us :-)
+am.max-cmdline-args := xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+
+# $(call am.xargs-map,FUNCTION,LIST)
+# ----------------------------------
+# Map the function $1 on the arguments $2, ensuring that each
+# call of $1 has at most 40 arguments.
+# This implementation is hacky, but the more elegant or "naive" ones
+# (based on recursion) proved to be ludicrously memory-hungry with
+# huge lists.
+# A drawback of this implementation is that am.xargs-map cannot be
+# recursively invoked, but that shouldn't matter for our use cases.
+# The extra $(strip) calls are only to allow clearer formatting.
+define am.xargs-map
+$(if $2,$(strip \
+  )$(eval $0.partial-args :=)$(strip \
+  )$(eval $0.counter :=)$(strip \
+  )$(foreach i,$2,$(strip \
+    )$(eval $0.counter := $$($0.counter)x)$(strip \
+    )$(eval $0.partial-args += $$i)$(strip \
+    )$(if $(filter $(am.max-cmdline-args),$($0.counter)),$(strip \
+      )$(call $1,$(strip $($0.partial-args)))$(strip \
+      )$(eval $0.partial-args :=)$(strip \
+      )$(eval $0.counter :=)))$(strip \
+  )$(if $($0.counter),$(call $1,$(strip $($0.partial-args)))))
+endef
+
 ## Some derived variables that have been found to be useful.
 pkgdatadir = $(datadir)/@PACKAGE@
 pkgincludedir = $(includedir)/@PACKAGE@
diff --git a/t/am-xargs-map.sh b/t/am-xargs-map.sh
new file mode 100755
index 0000000..4da959f
--- /dev/null
+++ b/t/am-xargs-map.sh
@@ -0,0 +1,139 @@
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Test Automake internal function 'am.xargs-map', in several usage
+# scenarios.
+
+am_create_testdir=empty
+. ./defs || exit 1
+
+cp "$am_amdir"/header-vars.am . \
+  || fatal_ "fetching makefile fragment headers-vars.am"
+
+# Filter out Automake comments and things that would need configure
+# substitutions.
+LC_ALL=C $EGREP -v '(^##|address@hidden@)' header-vars.am > defn.mk
+rm -f header-vars.am
+
+sed 's/^[0-9][0-9]*:://' > Makefile << 'END'
+01::include ./defn.mk
+02::
+03::args1  := 0 1 2 3 4 5 6 7 8 9
+04::args2  := $(args1) $(args1)
+05::args4  := $(args2) $(args2)
+06::args8  := $(args4) $(args4)
+07::args16 := $(args8) $(args8)
+08::
+09::WARN := no
+10::ifeq ($(WARN),yes)
+11::  $(call am.xargs-map,warning,$(args16))
+12::  $(call am.xargs-map,warning,$(args16) 0 1 2 3)
+13::  $(call am.xargs-map,warning,x y z)
+14::endif
+
+args32 := $(args16) $(args16)
+args64 := $(args32) $(args32)
+
+bar = test '$1' = '$(args4)'$(am__newline)
+test-xargs-map:
+       $(call am.xargs-map,bar,$(args16))
+
+args = $(error 'args' should be overridden from the command line)
+foo = @echo $1$(am__newline)
+echo-xargs-map:
+       $(call am.xargs-map,foo,$(args))
+END
+
+args1="0 1 2 3 4 5 6 7 8 9"
+args2="$args1 $args1"
+args4="$args2 $args2"
+
+$MAKE .am/nil WARN=yes 2>stderr || { cat stderr >&2; exit 1; }
+cat stderr >&2
+grep '^Makefile:' stderr # For debugging
+test $(grep -c "^Makefile:11: $args4$" stderr) -eq 4
+test $(grep -c "^Makefile:12: $args4$" stderr) -eq 4
+test $(grep -c "^Makefile:12: 0 1 2 3$" stderr) -eq 1
+test $(grep -c "^Makefile:13: x y z$" stderr) -eq 1
+test $(grep -c "^Makefile:" stderr) -eq 10
+
+$MAKE 'test-xargs-map'
+
+check_echo ()
+{
+  cat > exp
+  $MAKE --no-print-directory "echo-xargs-map" args="$1" >got \
+    || { cat got >&2; exit 1; }
+  cat exp && cat got && diff exp got || exit 1
+}
+
+echo "$args1" | check_echo '$(args1)'
+echo "$args2" | check_echo '$(args2)'
+echo "$args4" | check_echo '$(args4)'
+
+check_echo '$(args8)'<<END
+$args4
+$args4
+END
+
+check_echo "$args4 $args4" <<END
+$args4
+$args4
+END
+
+check_echo "$args4 $args4 x" <<END
+$args4
+$args4
+x
+END
+
+check_echo "$args4 01 02 03 04 05 06 07" <<END
+$args4
+01 02 03 04 05 06 07
+END
+
+check_echo '$(args32) 11 12 13 67' <<END
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+11 12 13 67
+END
+
+check_echo '$(args64)' <<END
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+$args4
+END
+
+:
diff --git a/t/clean-many.sh b/t/clean-many.sh
new file mode 100755
index 0000000..3f3d9a7
--- /dev/null
+++ b/t/clean-many.sh
@@ -0,0 +1,63 @@
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that our cleaning rules do not hit errors due to a huge number
+# of files to be removed.  Checks similar in spirit are done by sister
+# test 'clean-many2.sh', which fakes an artificially low command line
+# length limit for 'rm'.
+
+. ./defs || exit 1
+
+cat >> configure.ac << 'END'
+AC_SUBST([safe_include], [include])
+AC_OUTPUT
+END
+
+cat > Makefile.am <<'END'
+# Don't use 'include' here: we want this processed at make runtime,
+# not at automake runtime.  Otherwise Automake runs too slow.
address@hidden@ files.mk
+MOSTLYCLEANFILES = $(files)
+CLEANFILES = $(files)
+DISTCLEANFILES = $(files)
+MAINTANERCLEANFILES = $(files)
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+./configure
+
+# Yes, we want to clean ~ 130 thousands files.  So what?
+i=2
+echo foo bar > t
+while test $i -le 17; do
+  i=$(($i + 1))
+  cat t t > t2
+  mv -f t2 t
+done
+(echo 'files = \' && sed 's/$/ \\/' t && echo '$(am__empty)') > files.mk
+rm -f t
+
+# 2^17 + 2 = 131074.
+test $(wc -l <files.mk) -eq 65538 || fatal_ "populating 'files.mk'"
+
+touch foo bar
+$MAKE maintainer-clean
+test ! -e foo
+test ! -e bar
+
+:
diff --git a/t/clean-many2.sh b/t/clean-many2.sh
new file mode 100755
index 0000000..e7505be
--- /dev/null
+++ b/t/clean-many2.sh
@@ -0,0 +1,81 @@
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that our cleaning rules do not hit errors due to an exceeded
+# command line length when there are many files to clean.  Here, we
+# fake a very low command line length limit for 'rm' (max 50 arguments).
+# The sister test 'clean-many.sh' try to hit the real command line length
+# limit of the system, by declaring a huge number of files to be cleaned.
+
+. ./defs || exit 1
+
+echo AC_OUTPUT >> configure.ac
+
+oPATH=$PATH; export oPATH
+mkdir bin
+cat > bin/rm <<'END'
+#!/bin/sh
+PATH=$oPATH; export PATH
+if test $# -eq 0; then
+  echo "rm: missing argument" >&2
+  exit 1
+elif test $# -gt 50; then
+  echo "rm: argument list too long ($# arguments)" >&2
+  exit 1
+fi
+exec rm "$@"
+END
+chmod a+x bin/rm
+PATH=$(pwd)/bin$PATH_SEPARATOR$PATH; export PATH
+
+cat > Makefile.am <<'END'
+MOSTLYCLEANFILES = $(files)
+CLEANFILES = $(files)
+DISTCLEANFILES = $(files)
+MAINTANERCLEANFILES = $(files)
+include files.am
+END
+
+echo foo bar > t
+i=1
+while test $i -le 10; do
+  cat t t > t2 && mv -f t2 t || exit 99
+  i=$(($i + 1))
+done
+
+# Sanity check.
+rm -f $(cat t) && fatal_ "setting up 'rm' wrapper"
+
+(echo 'files = \' && sed 's/$/ \\/' t && echo '$(am__empty)') >files.am
+rm -f t
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+./configure
+cp config.status config-status.sav \
+
+for target in mostlyclean clean distclean maintainer-clean; do
+  echo dummy > foo
+  echo dummy > bar
+  $MAKE $target
+  test ! -e foo
+  test ! -e foo
+  cp config-status.sav config.status && ./config.status Makefile \
+    || fatal_ "resetting Makefile after cleanup"
+done
+
+:
diff --git a/t/internals.tap b/t/internals.tap
index aaf6b93..1012be6 100755
--- a/t/internals.tap
+++ b/t/internals.tap
@@ -19,7 +19,7 @@
 am_create_testdir=empty
 . ./defs || exit 1
 
-plan_ 8
+plan_ 10
 
 cp "$am_amdir"/header-vars.am . \
   || fatal_ "fetching makefile fragment headers-vars.am"
@@ -162,12 +162,27 @@ test-canonicalize:
        test '$(call am__canon,$(bslash)$(comma))' = '__'
        test '$(call am__canon,x$(comma)@$(bslash))' = 'address@hidden'
 
-.PHONY: test-newline
-test-newline:
-       @echo OK > foo$(am__newline)@touch bar$(am__newline)-false > baz
-       test `cat foo` = OK
-       test -f bar
-       test -f baz
+.PHONY: test-newline-1
+test-newline-1:
+       @echo OK > fo1$(am__newline)@touch ba1$(am__newline)-false > qu1
+       test `cat fo1` = OK
+       test -f ba1
+       test -f qu1
+
+my_newline = $(am__newline)
+.PHONY: test-newline-2
+test-newline-2:
+       @echo OK > fo2$(my_newline)@touch ba2$(my_newline)-false > qu2
+       test `cat fo2` = OK
+       test -f ba2
+       test -f qu2
+
+command-1 = test x = x$(am__newline)
+command-2 = test y = y$(my_newline)
+.PHONY: test-newline-3
+test-newline-3:
+       $(command-1)$(command-2)touch n3
+       test -f n3
 END
 
 command_ok_  am__strip_firstword        $MAKE test-strip-firstword
@@ -177,6 +192,8 @@ command_ok_  am__test_strip_suffixes    $MAKE 
test-strip-suffixes
 command_ok_  am__tolower                $MAKE test-tolower
 command_ok_  am__toupper                $MAKE test-toupper
 command_ok_  am__canon                  $MAKE test-canonicalize
-command_ok_  am__newline                $MAKE test-newline
+command_ok_  "am__newline (1)"          $MAKE test-newline-1
+command_ok_  "am__newline (2)"          $MAKE test-newline-2
+command_ok_  "am__newline (3)"          $MAKE test-newline-3
 
 :


hooks/post-receive
-- 
GNU Automake