Re: M4 syntax $11 vs. ${11}

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to Eric Blake on 1/19/2007 7:24 AM:
> According to Ralf Wildenhues on 1/18/2007 10:35 AM:
> [ Can we limit this to, say, the m4-discuss and autoconf lists? ]

Actually, this particular mail can probably stay on just autoconf-patches.

> 
>>> Anyways, here are the patches in their current state:
>> Please resubmit without any white-space only changes (e.g., diff -w).
>> It makes evaluating the amount of changes more difficult than necessary.
> 
> Coming in a later email; the machine I'm typing this mail on is not where
> I generated the patch.

Here's an updated (and smaller) patch that makes CVS autoconf happy with
branch-1_4 of M4, after today's commit that adds --warn-syntax to M4.

2007-01-27  Eric Blake  <address@hidden>

        * lib/m4sugar/m4sugar.m4: Cripple ${1} expansion in M4 2.0 for
        portability with 1.4.x behavior.
        * doc/autoconf.texi (Quoting and Parameters): New section.
        (Quotation and Nested Macros): Touch up example.
        * NEWS: Document the new limitation on user macros not exceeding
        $9.
        * lib/m4sugar/m4sh.m4 (_AS_BOURNE_COMPATIBLE)
        (_AS_DETECT_BETTER_SHELL): Avoid failure when using M4 1.4.9's
        --warn-syntax option.
        * lib/autoconf/programs.m4 (_AC_CHECK_PROG): Likewise.
        * lib/autotest/general.m4 (_AT_CREATE_DEBUGGING_SCRIPT):
        Likewise.

- --
Don't work too hard, make some time for fun as well!

Eric Blake             address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFvC8p84KuGfSFAYARAtIcAKDNWerjQUt2kmKBYj14y1EHiw05MQCglcsW
xJ/N5/h+BX5LFSjIl5XMY5c=
=fj1C
-----END PGP SIGNATURE-----

Index: NEWS
===================================================================
RCS file: /sources/autoconf/autoconf/NEWS,v
retrieving revision 1.422
diff -u -p -r1.422 NEWS
--- NEWS        19 Dec 2006 05:41:18 -0000      1.422
+++ NEWS        28 Jan 2007 04:04:51 -0000
@@ -5,6 +5,9 @@
    These warnings can be disabled with the new AC_DISABLE_OPTION_CHECKING
    macro, or by invoking 'configure' with --disable-option-checking.
 
+** For portability with the eventual M4 2.0, macros should no longer use
+   anything larger than $9 to refer to arguments.
+
 * Major changes in Autoconf 2.61a (2006-12-11)
 
 ** AC_FUNC_FSEEKO was broken in 2.61; it didn't make fseeko and ftello visible
Index: doc/autoconf.texi
===================================================================
RCS file: /sources/autoconf/autoconf/doc/autoconf.texi,v
retrieving revision 1.1128
diff -u -p -r1.1128 autoconf.texi
--- doc/autoconf.texi   23 Jan 2007 18:47:23 -0000      1.1128
+++ doc/autoconf.texi   28 Jan 2007 04:04:58 -0000
@@ -434,6 +434,7 @@ M4 Quotation
 
 * Active Characters::           Characters that change the behavior of M4
 * One Macro Call::              Quotation and one macro call
+* Quoting and Parameters::      M4 vs. shell parameters
 * Quotation and Nested Macros::  Macros calling macros
 * Changequote is Evil::         Worse than INTERCAL: M4 + changequote
 * Quadrigraphs::                Another way to escape special characters
@@ -8860,10 +8861,6 @@ and their interface might change in the 
 @cindex M4 quotation
 @cindex quotation
 
address@hidden FIXME: Grmph, yet another quoting myth: quotation has *never*
address@hidden prevented `expansion' of $1.  Unless it refers to the expansion
address@hidden of the value of $1?  Anyway, we need a rewrite address@hidden
-
 The most common problem with existing macros is an improper quotation.
 This section, which users of Autoconf can skip, but which macro writers
 @emph{must} read, first justifies the quotation scheme that was chosen
@@ -8873,6 +8870,7 @@ former helps one to follow the latter.
 @menu
 * Active Characters::           Characters that change the behavior of M4
 * One Macro Call::              Quotation and one macro call
+* Quoting and Parameters::      M4 vs. shell parameters
 * Quotation and Nested Macros::  Macros calling macros
 * Changequote is Evil::         Worse than INTERCAL: M4 + changequote
 * Quadrigraphs::                Another way to escape special characters
@@ -8886,8 +8884,8 @@ To fully understand where proper quotati
 to know what the special characters are in Autoconf: @samp{#} introduces
 a comment inside which no macro expansion is performed, @samp{,}
 separates arguments, @samp{[} and @samp{]} are the quotes themselves,
-and finally @samp{(} and @samp{)} (which M4 tries to match by
-pairs).
address@hidden(} and @samp{)} (which M4 tries to match by pairs), and finally
address@hidden inside a macro definition.
 
 In order to understand the delicate case of macro calls, we first have
 to present some obvious failures.  Below they are ``obvious-ified'',
@@ -9008,10 +9006,73 @@ car([[[a]]])
 @result{}[a]
 @end example
 
address@hidden Quoting and Parameters
address@hidden
+
+When M4 encounters @samp{$} within a macro definition, followed
+immediately by a character it recognizes, it will perform M4 parameter
+expansion.  This happens regardless of how many layers of quotes the
+parameter expansion is nested within, or even if it occurs in text that
+will be rescanned as a comment.
+
address@hidden
+define([none], [$1])
address@hidden
+define([one], [[$1]])
address@hidden
+define([two], [[[$1]]])
address@hidden
+define([comment], [# $1])
address@hidden
+define([active], [ACTIVE])
address@hidden
+none([active])
address@hidden
+one([active])
address@hidden
+two([active])
address@hidden
+comment([active])
address@hidden active
address@hidden example
+
+On the other hand, since autoconf generates shell code, you often want
+to output shell variable expansion, rather than performing M4 parameter
+expansion.  To do this, you must use M4 quoting to separate the @samp{$}
+from the next character in the definition of your macro.  If the macro
+definition occurs in single-quoted text, then insert another level of
+quoting; if the usage is already inside a double-quoted string, then
+split it into concatenated strings.
+
address@hidden
+define([single], [one set of quotes for $[]1 in definition])
address@hidden
+define([double], [[two sets of quotes for $][1 in definition]])
address@hidden
+single
address@hidden set of quotes for $1 in definition
+double
address@hidden sets of quotes for $1 in definition
address@hidden example
+
address@hidden states that M4 implementations are free to use
address@hidden@{} in macro definitions however they like.  @acronym{GNU} M4
+1.4.x outputs those two characters literally, whereas @acronym{GNU} M4
+2.0 has an optional extension modeled after shell variable expansion
+semantics.  This is in part due to the fact that M4 1.4.x violates
address@hidden by treating @samp{$11} as the eleventh parameter, rather
+than the first parameter concatenated with the string @samp{1}.  To
+comply with @acronym{POSIX}, M4 2.0 prefers @address@hidden@}} rather than
address@hidden when referring to the eleventh parameter.  In Autoconf, the
+M4 2.0 extensions with @address@hidden are disabled, so you can safely use
address@hidden@address@hidden in your macro definitions and have it result in 
literal
+output, without worrying that M4 will treat it like @samp{$1}.  However,
+it does mean that you cannot refer to anything higher than @samp{$9} in
+your macro, if you expect to be portable to both M4 1.4.x and 2.0.
+
 With this in mind, we can explore the cases where macros invoke
 address@hidden
 
-
 @node Quotation and Nested Macros
 @subsection Quotation and Nested Macros
 
@@ -9105,17 +9166,20 @@ qar([int tab[10];])
 @noindent
 Ahhh!  That's much better.
 
-But note what you've done: now that the arguments are literal strings,
-if the user wants to use the results of expansions as arguments, she has
-to use an @emph{unquoted} macro call:
+But note what you've done: now that the result of @code{qar} is always
+a literal string, the only time a user can use the results of
+expansions is with an @emph{unquoted} macro call:
 
 @example
 qar(active)
 @result{}ACT
+qar([active])
address@hidden
 @end example
 
 @noindent
-where she wanted to reproduce what she used to do with @code{car}:
+leaving no way for the user to reproduce what she used to do with
address@hidden:
 
 @example
 car([active])
Index: lib/autoconf/programs.m4
===================================================================
RCS file: /sources/autoconf/autoconf/lib/autoconf/programs.m4,v
retrieving revision 1.65
diff -u -p -r1.65 programs.m4
--- lib/autoconf/programs.m4    23 Jan 2007 18:54:38 -0000      1.65
+++ lib/autoconf/programs.m4    28 Jan 2007 04:04:58 -0000
@@ -2,7 +2,7 @@
 # Checking for programs.
 
 # Copyright (C) 1992, 1993, 1994, 1995, 1996, 1998, 1999, 2000, 2001,
-# 2002, 2003, 2004, 2005, 2006 Free Software Foundation, Inc.
+# 2002, 2003, 2004, 2005, 2006, 2007 Free Software Foundation, Inc.
 
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -93,7 +93,7 @@ m4_ifvaln([$6],
     # However, it has the same basename, so the bogon will be chosen
     # first if we set $1 to just the basename; use the full file name.
     shift
-    ac_cv_prog_$1="$as_dir/$ac_word${1+' 'address@hidden"
+    ac_cv_prog_$1="$as_dir/$ac_word[$]{1+' 'address@hidden"
 m4_if([$2], [$4],
 [  else
     # Default is a loser.
Index: lib/autotest/general.m4
===================================================================
RCS file: /sources/autoconf/autoconf/lib/autotest/general.m4,v
retrieving revision 1.218
diff -u -p -r1.218 general.m4
--- lib/autotest/general.m4     25 Nov 2006 09:57:34 -0000      1.218
+++ lib/autotest/general.m4     28 Jan 2007 04:04:58 -0000
@@ -1,8 +1,8 @@
 # This file is part of Autoconf.                          -*- Autoconf -*-
 # M4 macros used in building test suites.
 
-# Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software
-# Foundation, Inc.
+# Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007 Free
+# Software Foundation, Inc.
 
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -169,9 +169,9 @@ m4_define([_AT_NORMALIZE_TEST_GROUP_NUMB
 m4_define([_AT_CREATE_DEBUGGING_SCRIPT],
 [        {
            echo "#! /bin/sh"
-           echo 'test "${ZSH_VERSION+set}" = set && alias -g 
'\''${1+"address@hidden"}'\''='\''"address@hidden"'\'''
+           echo 'test "${ZSH_VERSION+set}" = set && alias -g 
'\''[$]{1+"address@hidden"}'\''='\''"address@hidden"'\'''
            AS_ECHO(["cd '$at_dir'"])
-           AS_ECHO(["exec \${CONFIG_SHELL-$SHELL} $[0] -v -d $at_debug_args 
$at_group \${1+\"address@hidden"}"])
+           AS_ECHO(["exec \${CONFIG_SHELL-$SHELL} $[0] -v -d $at_debug_args 
$at_group \[$]{1+\"address@hidden"}"])
            echo 'exit 1'
          } >$at_group_dir/run
          chmod +x $at_group_dir/run
Index: lib/m4sugar/m4sh.m4
===================================================================
RCS file: /sources/autoconf/autoconf/lib/m4sugar/m4sh.m4,v
retrieving revision 1.204
diff -u -p -r1.204 m4sh.m4
--- lib/m4sugar/m4sh.m4 1 Dec 2006 18:32:35 -0000       1.204
+++ lib/m4sugar/m4sh.m4 28 Jan 2007 04:04:59 -0000
@@ -2,7 +2,7 @@
 # M4 sugar for common shell constructs.
 # Requires GNU M4 and M4sugar.
 #
-# Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software
+# Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007 Free Software
 # Foundation, Inc.
 #
 # This program is free software; you can redistribute it and/or modify
@@ -180,9 +180,9 @@ m4_define([_AS_BOURNE_COMPATIBLE],
 [AS_IF([test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1],
  [emulate sh
   NULLCMD=:
-  [#] Zsh 3.x and 4.x performs word splitting on ${1+"address@hidden"}, which
+  [#] Zsh 3.x and 4.x performs word splitting on [$]{1+"address@hidden"}, which
   # is contrary to our usage.  Disable this feature.
-  alias -g '${1+"address@hidden"}'='"address@hidden"'
+  alias -g '[$]{1+"address@hidden"}'='"address@hidden"'
   setopt NO_GLOB_SUBST],
  [AS_CASE([`(set -o) 2>/dev/null`], [*posix*], [set -o posix])])
 ])
@@ -273,7 +273,7 @@ if test "x$CONFIG_SHELL" = x; then
         do ($as_unset $as_var) >/dev/null 2>&1 && $as_unset $as_var
         done
         export CONFIG_SHELL
-        exec "$CONFIG_SHELL" "$as_myself" ${1+"address@hidden"}])
+        exec "$CONFIG_SHELL" "$as_myself" [$]{1+"address@hidden"}])
 
     AS_IF([test $as_have_required = no],
       [echo This script requires a shell more modern than all the
Index: lib/m4sugar/m4sugar.m4
===================================================================
RCS file: /sources/autoconf/autoconf/lib/m4sugar/m4sugar.m4,v
retrieving revision 2.100
diff -u -p -r2.100 m4sugar.m4
--- lib/m4sugar/m4sugar.m4      20 Oct 2006 01:34:33 -0000      2.100
+++ lib/m4sugar/m4sugar.m4      28 Jan 2007 04:04:59 -0000
@@ -3,8 +3,8 @@ divert(-1)#                             
 # Base M4 layer.
 # Requires GNU M4.
 #
-# Copyright (C) 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software
-# Foundation, Inc.
+# Copyright (C) 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007 Free
+# Software Foundation, Inc.
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -54,6 +54,12 @@ divert(-1)#                             
 # Set the quotes, whatever the current quoting system.
 changequote()
 changequote([, ])
+# M4 2.0 treats ${1} as an expansion of parameter one, but too many
+# existing autoconf macros expect the 1.4.x behavior of literal output.
+# We can tell the difference by whether the changesyntax builtin exists;
+# this method cripples the 2.0 extension, but does mean that anything
+# higher than $9 cannot be used in macro definitions.
+ifdef([changesyntax], [changesyntax([O+{}])])
 
 # Some old m4's don't support m4exit.  But they provide
 # equivalent functionality by core dumping because of the