planning for beta-testing gnulib-tool.py

[Bruno Haible]

As the closing of gaps in the Python implementation of gnulib-tool progresses,
and as Collin Funk reports today:

  these changes (aside
  from the issues you have mentioned) are enough to bootstrap Coreutils.

it is time to think about how to manage the beta-testing of gnulib-tool.py
and the switch from the shell to the Python implementation.

The big problem there is:
  - We don't have a test suite for gnulib-tool. Nothing. Nada.
  - Users who have been using gnulib-tool for 15 years don't want to
    face breakage now.

Here are some thoughts about it. Comments welcome.

A. Planning for the beta-test
=============================

1) First we need to complete the gnulib-tool.py.TODO. It does not make sense
   to invite users to switch to the Python implementation while it is incomplete
   or while we are still making significant optimizations on it.

2) Then, I would propose to 'git mv gnulib-tool gnulib-tool.sh' and create
   a new script gnulib-tool, that invokes either gnulib-tool.sh or
   gnulib-tool.py, with the same arguments. It does so based on an environment
   variable GNULIB_TOOL_IMPL with value 'py' or 'sh'. If this environment
   variable is not set, it invokes gnulib-tool.sh.

   Since we dispatch according to an environment variable, gnulib-tool
   invocations via 'bootstrap' can we influenced without modifying neither
   'bootstrap' nor 'bootstrap.conf'.

3) After having _ourselves_ made sure that both implementations produce
   identical results (that is, identical files on disk and identical stdout
   output) on a certain package, we can reasonably tell the maintainers of
   that package: "we believe you can set GNULIB_TOOL_IMPL=py and enjoy a
   faster gnulib-tool".

4) There is a list of packages that use gnulib, in the file users.txt.
   We can pick, say, the 10 most important or most active GNU packages among
   these, and repeat step 3) for each of them.

5) Possibly it makes also sense to allow GNULIB_TOOL_IMPL to be set to
   'sh+py'. In this case the script will make a full copy of the destination
   dir, run the shell implementation and the Python implementation on the
   two destination dirs, separately, and compare the results (again, both
   in terms of effects on the file system, as well as standard output).
   And err out if they are different.

6) Then, when we are confident enough, we would make GNULIB_TOOL_IMPL=py
   the default, instead of GNULIB_TOOL_IMPL=sh.

B. Constructing a test suite
============================

Long term (20+ years), neither Paul nor I may be very active any more,
since we are both getting old. It is therefore important to enable
further maintenance by having a test suite.

A single test case (for --import, which is the main case) consists of
a package with configure.ac and Makefile.ams, a gnulib-tool invocation,
and an expected output.

It is obvious that a test suite will be huge (in the range of > 100 MB,
possibly 1 GB). Therefore it should be stored in a separate git repository,
since many people use "git clone gnulib.git" either implicitly through
a submodule, or as part of a continuous integration, and they should
not be bothered downloading useless stuff.

But the test suite should also have at least one unit test for each
command-line option. That is quite some manual effort to write.


Bruno