Re: [PATCH] yacc: extend and improve tests

[Ralf Wildenhues]

Hi Stefano,

* Stefano Lattarini wrote on Sat, Jan 08, 2011 at 02:33:34PM CET:
> Here are some testsuite enhancements that might help the on-going
> work on Yacc support.  OK for a proper temporary branch off of
> 'yacc-clean' (after having merged into it the older temporary
> branch 'tests-lexyacc-extend' a.k.a. commit v1.11-249-gd4dcf50),
> to be merged in master afterwards?

Sure.  You decide where and when to merge this best.

OK with nits below addressed.

Thanks!
Ralf

> Subject: [PATCH] yacc: extend and improve tests
> 
> * tests/yacc-basic.test: Also check that the intermediate C file
> is mentioned in the generated Makefile.in, and that it is created
> by the first make invocation.
> * tests/yacc.test: Test removed, it has become obsolete now.

Is that true?  AFAICS the other tests all require 'bison' to be present,
while this one does not.

> * tests/yacc3.test: Test removed, superseded by ...
> * tests/yacc-d-basic.test: ... this new test.
> * tests/yacc2.test: Add reference to that new test in the heading
> comments.
> * tests/yacc-d-vpath.test: New test.
> * tests/yaccvpath.test: Updated heading comments.  Do not require
> gcc anymore, as any working C compiler should be enough.  Remove
> redundant comments.
> * tests/yacc-nodist.test: New test.
> * tests/yacc-dist-nobuild.test: New test.
> * tests/Makefile.am (TESTS): Update.

> --- a/tests/yacc-basic.test
> +++ b/tests/yacc-basic.test

> @@ -55,12 +59,18 @@ $AUTOMAKE -a
>  
>  ./configure
>  $MAKE
> +# The `parse.c' must be created and not removed (i.e., not treated

The `parse.c' file

> +# like an "intermediate file" in GNU make sense).

in the

> +test -f parse.c
>  
>  echo a | ./foo
>  echo b | ./foo && Exit 1
>  
>  # The generated file `parse.c' must be shipped.
> +$MAKE echo-distcom
> +$MAKE -s echo-distcom | grep '[ /]parse.c '
>  $MAKE distdir
> +ls -l $distdir
>  test -f $distdir/parse.c
>  
>  # Sanity check on distribution.

> --- /dev/null
> +++ b/tests/yacc-d-basic.test
> @@ -0,0 +1,157 @@
> +#! /bin/sh
> +# Copyright (C) 2011 Free Software Foundation, Inc.

If this supersedes yacc3.test, then does it not also derive from it?
I think in that case it is prudent to add the copyright years from that
file here as well.

> +# Tests on basic Yacc support for when we have -d in YFLAGS, AM_YFLAGS
> +# and maude_YFLAGS.

s/and/or/  ?

> +. ./defs || Exit 1

Doesn't this require bison?

> +set -e
> +
> +tab='        '
> +distdir=$me-1.0
> +
> +cat >> configure.in << 'END'
> +AC_PROG_CC
> +AC_PROG_YACC
> +AC_CONFIG_FILES([foo/Makefile bar/Makefile baz/Makefile])
> +AC_OUTPUT
> +END
> +
> +cat > Makefile.am <<'END'
> +SUBDIRS = foo bar baz
> +END
> +
> +mkdir foo bar baz
> +
> +cat > foo/Makefile.am <<'END'
> +bin_PROGRAMS = zardoz
> +zardoz_SOURCES = parse.y main.c
> +.PHONY: echo-distcom
> +echo-distcom:
> +     @echo ' ' $(DIST_COMMON) ' '
> +END
> +cp foo/Makefile.am bar/Makefile.am
> +cp foo/Makefile.am baz/Makefile.am
> +
> +cat > foo/parse.y << 'END'
> +%{
> +#include "parse.h"
> +int yylex () { return 0; }
> +void yyerror (char *s) {}
> +%}
> +%%
> +x : 'x' {};
> +%%
> +END
> +cp foo/parse.y bar/parse.y
> +
> +cat > foo/main.c << 'END'
> +#include "parse.h"
> +int main ()
> +{
> +  return yyparse ();
> +}
> +END
> +cp foo/main.c bar/main.c
> +
> +# Even the generated header file is renamed when target-specific YFLAGS
> +# are used.  This might not be the best semantic, but it has been in place

semantics

> +# for quite a long time, so just go along with it for now.

Just out of curiosity: can you explain why this may not be good
semantics?

> +sed 's/"parse\.h"/"zardoz-parse.h"/' foo/parse.y > baz/parse.y
> +sed 's/"parse\.h"/"zardoz-parse.h"/' foo/main.c > baz/main.c
> +
> +$ACLOCAL
> +$AUTOCONF
> +
> +$AUTOMAKE -a
> +$FGREP parse.h foo/Makefile.in bar/Makefile.in baz/Makefile.in && Exit 1
> +
> +cat >> foo/Makefile.am <<END
> +BUILT_SOURCES = parse.h
> +YFLAGS=\
> +-d
> +END
> +$AUTOMAKE -Wno-gnu foo/Makefile
> +
> +sed 's/EOL$//' >> bar/Makefile.am <<END
> +AM_YFLAGS${tab}=  -d EOL
> +BUILT_SOURCES = parse.h
> +END
> +$AUTOMAKE bar/Makefile
> +
> +cat >> baz/Makefile.am <<END
> +BUILT_SOURCES = zardoz-parse.h
> +zardoz_YFLAGS =-d${tab}
> +END
> +$AUTOMAKE baz/Makefile
> +
> +./configure
> +
> +$MAKE
> +
> +test -f foo/parse.c
> +test -f foo/parse.h
> +test -f bar/parse.c
> +test -f bar/parse.h
> +test -f baz/zardoz-parse.c
> +test -f baz/zardoz-parse.h
> +
> +# The generated C and header files must be shipped.
> +for dir in foo bar; do
> +  cd $dir
> +  $MAKE echo-distcom
> +  $MAKE -s echo-distcom | grep '[ /]parse.c '
> +  $MAKE -s echo-distcom | grep '[ /]parse.h '
> +  cd ..
> +done
> +cd baz
> +$MAKE echo-distcom
> +$MAKE -s echo-distcom | grep '[ /]zardoz-parse.c '
> +$MAKE -s echo-distcom | grep '[ /]zardoz-parse.h '

Did you try with some non-GNU make (e.g., heirloom or BSD make)?

> +cd ..
> +
> +$MAKE distdir
> +ls -l $distdir
> +test -f $distdir/foo/parse.c
> +test -f $distdir/foo/parse.h
> +test -f $distdir/bar/parse.c
> +test -f $distdir/bar/parse.h
> +test -f $distdir/baz/zardoz-parse.c
> +test -f $distdir/baz/zardoz-parse.h
> +
> +# Sanity check on distribution.

s/on //

> +$MAKE distcheck
> +
> +# While we are at it, make sure that `parse.c' and `parse.h' are erased
> +# by maintainer-clean, and not by distclean.
> +$MAKE distclean
> +test -f foo/parse.c
> +test -f foo/parse.h
> +test -f bar/parse.c
> +test -f bar/parse.h
> +test -f baz/zardoz-parse.c
> +test -f baz/zardoz-parse.h
> +./configure # We must re-create `Makefile'.
> +$MAKE maintainer-clean
> +test ! -f foo/parse.c
> +test ! -f foo/parse.h
> +test ! -f bar/parse.c
> +test ! -f bar/parse.h
> +test ! -f baz/zardoz-parse.c
> +test ! -f baz/zardoz-parse.h
> +
> +:

> --- /dev/null
> +++ b/tests/yacc-d-vpath.test
> @@ -0,0 +1,112 @@
> +#! /bin/sh
> +# Copyright (C) 2011 Free Software Foundation, Inc.

> +# This test checks that dependent files are updated before including

including them

> +# in the distribution. `parse.c' depends on `parse.y'. The later is

.  The latter

> +# updated so that `parse.c' should be rebuild. Then we are running

rebuilt.  Then we run

> +# `make' and `make distdir' and check whether the version of `parse.c'
> +# to be distributed is up to date.
> +
> +# Please keep this in sync with sister test `yaccvapth.test'.

yaccvpath?

> +
> +required=bison
> +. ./defs || Exit 1
> +
> +set -e
> +
> +distdir=$me-1.0
> +
> +cat >> configure.in << 'END'
> +AC_PROG_CC
> +AC_PROG_YACC
> +AC_OUTPUT
> +END
> +
> +cat > Makefile.am << 'END'
> +bin_PROGRAMS = foo
> +foo_SOURCES = parse.y foo.c
> +AM_YFLAGS = -d
> +END
> +
> +# Original parser, with `foobar'
> +cat > parse.y << 'END'
> +%{
> +int yylex () {return 0;}
> +void yyerror (char *s) {}
> +%}
> +%token FOOBAR
> +%%
> +foobar : 'f' 'o' 'o' 'b' 'a' 'r' {};
> +END
> +
> +cat > foo.c << 'END'
> +#include "parse.h"
> +int main () { return 0; }
> +END
> +
> +$ACLOCAL
> +$AUTOCONF
> +$AUTOMAKE -a
> +
> +$YACC -d parse.y
> +mv y.tab.c parse.c
> +mv y.tab.h parse.h
> +# Sanity checks.
> +grep foobar parse.c
> +grep FOOBAR parse.h
> +
> +mkdir sub
> +cd sub
> +../configure
> +
> +$sleep
> +
> +# New parser, with `fubar'
> +cat > ../parse.y << 'END'
> +%{
> +int yylex () {return 0;}
> +void yyerror (char *s) {}
> +%}
> +%token FUBAR
> +%%
> +fubar : 'f' 'o' 'o' 'b' 'a' 'r' {};

Is the RHS wrong on purpose here?

> +END
> +
> +$MAKE
> +$MAKE distdir
> +$FGREP fubar $distdir/parse.c
> +$FGREP FUBAR $distdir/parse.h
> +
> +# Now check to make sure that `make dist' will rebuild the parser.
> +
> +$sleep
> +
> +# New parser, with `maude'
> +cat > ../parse.y << 'END'
> +%{
> +int yylex () {return 0;}
> +void yyerror (char *s) {}
> +%}
> +%token MAUDE
> +%%
> +maude : 'm' 'a' 'u' 'd' 'e' {};
> +END
> +
> +$MAKE distdir
> +$FGREP maude $distdir/parse.c
> +$FGREP MAUDE $distdir/parse.h
> +
> +:

> --- /dev/null
> +++ b/tests/yacc-dist-nobuild.test
> @@ -0,0 +1,95 @@
> +#! /bin/sh

> +# This test checks that dependent files are updated before including
> +# in the distribution. `parse.c' depends on `parse.y'. The later is

.  The latter

> +# updated so that `parse.c' should be rebuild. Then we are running

rebuilt.  T

> +# `make' and `make distdir' and check whether the version of `parse.c'
> +# to be distributed is up to date.
> +
> +# Check that distributed Yacc-generated parsers are not uselessly
> +# remade from an unpacked distributed tarball.
> +
> +required=bison
> +. ./defs || Exit 1
> +
> +set -e
> +
> +distdir=$me-1.0
> +
> +cat >> configure.in << 'END'
> +AC_PROG_CC
> +AC_PROG_YACC
> +AC_OUTPUT
> +END
> +
> +cat > Makefile.am << 'END'
> +bin_PROGRAMS = foobar zardoz
> +foobar_SOURCES = parse.y main.c
> +zardoz_SOURCES = $(foobar_SOURCES)
> +zardoz_YFLAGS = -d
> +END
> +
> +cat > parse.y << 'END'
> +%{
> +int yylex () { return 0; }
> +void yyerror (char *s) {}
> +%}
> +%%
> +foobar : 'f' 'o' 'o' 'b' 'a' 'r' {};
> +END
> +
> +cat > main.c << 'END'
> +int main () { return 0; }
> +END
> +
> +$ACLOCAL
> +$AUTOCONF
> +$AUTOMAKE -a
> +
> +./configure
> +$MAKE
> +
> +$MAKE distdir
> +chmod -R a-w $distdir
> +
> +mkdir bin
> +cat > bin/yacc <<'END'
> +#!/bin/sh
> +echo "$0 invoked, shouldn't happen!" >&2
> +exit 1
> +END
> +cp bin/yacc bin/bison
> +chmod a+x bin/yacc bin/bison
> +PATH=`pwd`/bin$PATH_SEPARATOR$PATH
> +
> +YACC=yacc BISON=bison
> +export YACC BISON
> +
> +mkdir build
> +cd build
> +../$distdir/configure
> +$MAKE
> +
> +# Sanity check.
> +chmod u+w ../$distdir
> +rm -f ../$distdir/parse.c
> +chmod a-w ../$distdir
> +$MAKE >out 2>&1 && { cat out;  Exit 1; }

extra space.

> +cat out
> +$FGREP parse.c out
> +
> +:

> --- /dev/null
> +++ b/tests/yacc-nodist.test
> @@ -0,0 +1,103 @@

> +# Checks for .c and .h files derived from non-distributed .y sources.
> +
> +required=bison
> +. ./defs || Exit 1
> +
> +set -e
> +
> +cat >> configure.in << 'END'
> +AC_PROG_CC
> +AC_PROG_YACC
> +AC_CONFIG_FILES([sub1/Makefile sub2/Makefile])
> +AC_OUTPUT
> +END
> +
> +mv -f configure.in configure.stub
> +
> +cat > Makefile.am << 'END'
> +SUBDIRS = sub1 sub2
> +.PHONY: test
> +test-build: all
> +     ls -l . sub1 sub2
> +     test -f sub1/parse.y
> +     test -f sub1/parse.c
> +     test -f sub2/parse.y
> +     test -f sub2/parse.c
> +     test -f sub2/parse.h
> +test-dist: distdir
> +     ls -l $(distdir) $(distdir)/sub1 $(distdir)/sub2
> +     test ! -r $(distdir)/sub1/parse.y
> +     test ! -r $(distdir)/sub1/parse.c
> +     test ! -r $(distdir)/sub1/parse.h
> +     test ! -r $(distdir)/sub2/parse.y
> +     test ! -r $(distdir)/sub2/parse.c
> +     test ! -r $(distdir)/sub2/parse.h
> +check-local: test-build test-dist
> +END
> +
> +mkdir sub1 sub2
> +
> +cat > sub1/Makefile.am << 'END'
> +parse.y:
> +     rm -f $@ address@hidden
> +     { : \

Prepend ':;' to the command, please, for some bash version.

> +       && echo "%{" \
> +       && echo "int yylex () { return 0; }" \
> +       && echo "void yyerror (char *s) {}" \
> +       && echo "%}" \
> +       && echo "%%" \
> +       && echo "maude : 'm' 'a' 'u' 'd' 'e' {}"; \
> +     } > address@hidden
> +     chmod a-w address@hidden
> +     mv -f address@hidden $@
> +bin_PROGRAMS = prog
> +prog_SOURCES = main.c
> +nodist_prog_SOURCES = parse.y
> +CLEANFILES = $(nodist_prog_SOURCES)
> +END
> +
> +cat sub1/Makefile.am - > sub2/Makefile.am << 'END'
> +AM_YFLAGS = -d
> +BUILT_SOURCES = parse.h
> +END
> +
> +cat > sub1/main.c << 'END'
> +int main ()
> +{
> +  return yyparse ();
> +}
> +END
> +cat - sub1/main.c > sub2/main.c << 'END'
> +#include "parse.h"
> +END
> +
> +$ACLOCAL
> +$AUTOCONF
> +$AUTOMAKE -a
> +
> +./configure
> +$MAKE
> +$MAKE test-build
> +$MAKE test-dist
> +
> +# But the distribution must work correctly, assuming the user has
> +# the proper tools to process yacc files.
> +$MAKE distcheck
> +
> +:

> --- a/tests/yacc2.test
> +++ b/tests/yacc2.test

copyright?

> @@ -17,6 +17,7 @@
>  
>  # Test to make sure intermediate .h file is not generated nor removed
>  # if (AM_)?YFLAGS do not contain -d.  Requested by Jim Meyering.
> +# See also the related semantic test `yacc-d-basic.test'.
>  
>  . ./defs || Exit 1