Re: a saner bootstrap script

[Gary V. Vaughan]

Ping?

On 16 Oct 2011, at 12:50, Gary V. Vaughan wrote:
> Hi Jim,
> 
> On 16 Oct 2011, at 04:15, Jim Meyering wrote:
>> Gary V. Vaughan wrote:
>>> Is there anything else I can do to help you incorporate this, and the
>>> matching bootstrap.conf I wrote for you into coreutils now that the
>>> release is out?
>> 
>> Thanks for persevering.  I have just tried it.
> 
> Excellent, thanks for that, and for the feedback.
> 
>> Please make at least a token effort to minimize differences in
>> bootstrap.conf.  That makes it a lot easier to compare to the original.
> 
> Apologies, I was starting from the Zile bootstrap.conf that I wrote,
> and copying over the various variable settings from the old coreutils
> bootstrap.conf, and not editing the coreutils bootstrap.conf itself,
> so I didn't think about needing to run diff between branches.
> 
>> Here is a new version with fewer gratuitous white space changes and
>> that avoids changing the copyright header.
> 
> 
> Thanks, I've applied most of that, although I routinely use diff -Ebu
> to see differences in code so as not to form the habit of avoiding
> consistent formatting purely for the sake of making diff work without
> a couple of extra flags.
> 
> Also, if you're not opposed to the 'Written by Gary V. Vaughan' line
> I'd rather keep that, since I did port the old bootstrap.conf to work
> with my saner bootstrap script after all.  On the other hand, I'm far
> from rabid about it, so if it annoys you, or you feel that the work
> that came before outweighs my contribution enough to make it inaccurate,
> then feel free to remove it when (and if) you merge.
> 
> Also, I kept the timestamp header line so that the emacs variable
> settings from the bottom of my version of the file correctly update
> the header timestamp on save.  Similarly, feel free to change back to
> your non-timestamped header comment and remove the emacs goo from
> the bottom when you merge if you'd prefer.
> 
>> It looks like your coreutils working directory must have contained
>> an intl/ directory, but most people don't have that.  Since I don't
>> have one, I get lots of these when compiling in lib/:
>> 
>>   cc1: error: ../intl: No such file or directory [-Werror]
> 
> No intl/ directory here, and I can't reproduce your failure.  Did you
> copy my bootstrap and bootstrap.conf into your own working directory,
> or clone my github mirror?
> 
> Here's an edited typescript of running directly with a fresh clone of
> github (from the revision before patching bootstrap.conf whitespace as
> you requested, not that it should matter):
> 
> address@hidden/Devo
> ├!10154│^2=git clone address@hidden:gvvaughan/GNU-coreutils.git
> Cloning into GNU-coreutils...
> remote: Counting objects: 152310, done.
> remote: Compressing objects: 100% (37473/37473), done.
> remote: Total 152310 (delta 114614), reused 152219 (delta 114531)
> Receiving objects: 100% (152310/152310), 27.17 MiB | 36 KiB/s, done.
> Resolving deltas: 100% (114614/114614), done.
> 
> address@hidden/Devo
> ├!10155│^2=cd GNU-coreutils
> 
> address@hidden/Devo/GNU-coreutils                             │master│8887350│
> ├!10156│^2=gco gary/bootstrap
> Branch gary/bootstrap set up to track remote branch gary/bootstrap from 
> origin.
> Switched to a new branch 'gary/bootstrap'
> 
> address@hidden/Devo/GNU-coreutils                     │gary/bootstrap│030770c│
> ├!10157│^2=PATH=~/Devo/autotools-latest/bin:/usr/local/Cellar/gettext/0.18.1.1/bin:$PATH
>                               
> 
> address@hidden/Devo/GNU-coreutils                     │gary/bootstrap│030770c│
> ├!10158│^2=./bootstrap --gnulib-srcdir=../gnulib
> bootstrap: `./bootstrap' differs from `./gnulib/build-aux/bootstrap', 
> bootstrap: please consider adopting the canonical version from gnulib.
> bootstrap: getting translations into po/.reference for coreutils...   
> receiving file list ... done
> ./
> af.po
> [[...]]
> bootstrap: running: autopoint --force
> Copying file ABOUT-NLS               
> Copying file build-aux/config.rpath
> [[...]]
> bootstrap: running: git clone --reference '../gnulib' 
> 'git://git.sv.gnu.org/gnulib.git' 'gnulib'
> Cloning into gnulib...
> remote: Counting objects: 1476, done.
> remote: Compressing objects: 100% (522/522), done.
> remote: Total 1271 (delta 1108), reused 871 (delta 749)
> Receiving objects: 100% (1271/1271), 974.96 KiB | 8 KiB/s, done.
> Resolving deltas: 100% (1108/1108), completed with 154 local objects.
> bootstrap: running: git submodule init
> Submodule 'gnulib' (git://git.sv.gnu.org/gnulib.git) registered for path 
> 'gnulib'
> bootstrap: running: git submodule update
> Submodule path 'gnulib': checked out 
> '244794a7887f13d9cdb91fed96932cc479905b96'
> bootstrap: running: gnulib/gnulib-tool --no-changelog 
> --avoid=canonicalize-lgpl --avoid=dummy --with-tests --aux-dir=build-aux 
> --m4-base=m4 --lib=libcoreutils --local-dir=gl --makefile-name=gnulib.mk 
> --tests-base=gnulib-tests --no-libtool --symbolic --import acl alignof...
> Module list with included dependencies (indented):
>    accept
>    accept-tests
>  acl
>    acl-tests
> [[...]]
> bootstrap: edited `lib/Makefile.am' (backup in `lib/Makefile.am~')
> bootstrap: running: autoreconf --symlink --install
> [[...]]
> bootstrap: linking file `gnulib/build-aux/compile'                    
> bootstrap: linking file `gnulib/build-aux/install-sh'                 
> bootstrap: linking file `gnulib/build-aux/missing'                    
> bootstrap: linking file `gnulib/build-aux/mdate-sh'                   
> bootstrap: linking file `gnulib/build-aux/texinfo.tex'                
> bootstrap: linking file `gnulib/build-aux/depcomp'                    
> bootstrap: linking file `gnulib/build-aux/config.guess'               
> bootstrap: linking file `gnulib/build-aux/config.sub'                 
> bootstrap: linking file `gnulib/doc/INSTALL'                          
> bootstrap: Please set copyright_holder explicitly in `bootstrap.conf';
> bootstrap: defaulting to `Free Software Foundation, Inc.'.            
> bootstrap: Creating po/Makevars from po/Makevars.template ...         
> bootstrap: If your pofiles are up-to-date, you can rerun bootstrap    
> bootstrap: as `bootstrap --skip-po' to avoid redownloading.           
> bootstrap: Done.  Now you can run './configure'.
> 
> address@hidden/Devo/GNU-coreutils                   *?│gary/bootstrap│030770c│
> ├!10160│^2=find . -name '*intl*' |grep -v /gnulib/
> ./m4/intl.m4~
> ./m4/intldir.m4~
> ./m4/intlmacosx.m4
> ./m4/intlmacosx.m4~                                                           
>                                         
> 
> address@hidden/Devo/GNU-coreutils                   *?│gary/bootstrap│030770c│
> ├!10161│^2=./configure
> checking for a BSD-compatible install... /usr/bin/install -c     
> checking whether build environment is sane... yes                
> checking for a thread-safe mkdir -p... build-aux/install-sh -c -d
> [[...]]
> configure: creating ./config.status                      
> config.status: creating Makefile                         
> config.status: creating doc/Makefile                     
> config.status: creating lib/Makefile                     
> config.status: creating man/Makefile                     
> config.status: creating po/Makefile.in                   
> config.status: creating src/Makefile                     
> config.status: creating tests/Makefile                   
> config.status: creating gnulib-tests/Makefile            
> config.status: creating lib/config.h                     
> config.status: executing depfiles commands               
> config.status: executing po-directories commands         
> config.status: creating po/POTFILES                      
> config.status: creating po/Makefile                      
> 
> address@hidden/Devo/GNU-coreutils                   *?│gary/bootstrap│030770c│
> ├!10162│^2=make
>  GEN    .version
> make  all-recursive
> Making all in lib
>  GEN    alloca.h
>  GEN    c++defs.h
>  GEN    warn-on-use.h
>  GEN    arg-nonnull.h
>  GEN    arpa/inet.h
>  GEN    configmake.h
> mv -f configmake.h-t configmake.h
>  GEN    ctype.h
>  GEN    dirent.h
>  GEN    fcntl.h
>  GEN    fnmatch.h
>  GEN    getopt.h
>  GEN    iconv.h
> gperf -m 10 ./iconv_open-aix.gperf > ./iconv_open-aix.h-t
> mv ./iconv_open-aix.h-t ./iconv_open-aix.h
> [[...]]
>  GEN    yes.1
> Making all in po
> Making all in tests
> make[2]: Nothing to be done for `all'.
> Making all in gnulib-tests
>  GEN    arg-nonnull.h
>  GEN    c++defs.h
>  GEN    unused-parameter.h
>  GEN    warn-on-use.h
> make  all-recursive
> Making all in .
> make[4]: Nothing to be done for `all-am'.
> make[2]: Nothing to be done for `all-am'.
> 
> address@hidden/Devo/GNU-coreutils                   *?│gary/bootstrap│030770c│
> ├!10163│^2=make check
>  GEN    public-submodule-commit
> make  check-recursive
> Making check in lib
> make  check-am
> [[...]]
> FAIL: misc/tty-eof (exit: 1)
> ============================
> 
> F: 1:
> F: 1: a b
> F: 1:
> F: 1:
> F: 1:
> F: 1: a b
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> F: 1:
> tty-eof: unexpand didn't exit after ^D from standard input
> F: 1:
> F: 1:
> F: 1:
> 
> [[...]]
> ======================================
> 1 of 369 tests failed
> (91 tests were not run)
> See tests/test-suite.log
> Please report to address@hidden
> ======================================
> make[4]: *** [test-suite.log] Error 1
> make[3]: *** [check-TESTS] Error 2
> make[2]: *** [check-am] Error 2
> make[1]: *** [check-recursive] Error 1
> make: *** [check] Error 2
> 
> The test failure is the same one I get when bootstrapped with the
> incumbent script.
> 
>> Merely creating the directory gets past that, and the build
>> did complete, but obviously I don't want to have to do that.
>> I used AM_GNU_GETTEXT([external]... after all.
>> 
>> Not surprisingly, the part of the current bootstrap script that
>> takes care of that transformation is absent from your version:
>> 
>> -      if test $file = Makefile.am && test "X$gnulib_mk" != XMakefile.am; 
>> then
>> -        copied=$copied${sep}$gnulib_mk; sep=$nl
>> -        remove_intl='/^[^#].*\/intl/s/^/#/;'"s!$bt_regex/!!g"
>> -        sed "$remove_intl" $1/$dir/$file |
>> -        cmp - $dir/$gnulib_mk > /dev/null || {
>> -          echo "$me: Copying $1/$dir/$file to $dir/$gnulib_mk ..." &&
>> -          rm -f $dir/$gnulib_mk &&
>> -          sed "$remove_intl" $1/$dir/$file >$dir/$gnulib_mk &&
>> -          gnulib_mk_hook $dir/$gnulib_mk
>> -        }
>> -      elif { test "${2+set}" = set && test -r $2/$dir/$file; } ||
>> -           version_controlled_file $dir $file; then
>> -        echo "$me: $dir/$file overrides $1/$dir/$file"
>> -      else
>> -        copied=$copied$sep$file; sep=$nl
>> -        cp_mark_as_generated $1/$dir/$file $dir/$file
>> -      fi || exit
>> 
> 
> As far as I can see, that code is not relevant except when creating and
> merging two directories... something that I've not done with my bootstrap
> and which is responsible for the bulk of the time savings compared to the
> incumbent bootstrap script.  However, slurp() is more or less uncommented,
> and even after staring at it for several months, I'm still not sure I
> properly understand it, so I could easily be wrong.  Nonetheless, my saner
> bootstrap seems to be working correctly on coreutils without it as far is
> I'm able to check.  What am I missing?
> 
> If there is a bug in gnulib-tool, or autopoint that puts unnecessary
> 'intl/' references into Makefiles when the presence of
> AM_GNU_GETTEXT_VERSION in configure.ac is a declaration that says there
> is no need for local 'intl/', then I think the bug should be fixed there
> rather than patched around in bootstrap.  Unfortunately, some difference
> in our environments is preventing me from exploring further since I can't
> reproduce your error on my machine :(
> 
> Are you running on Linux? I could build a Linux VM and try to reproduce
> it again there if you don't have time to investigate it yourself.

Cheers,
-- 
Gary V. Vaughan (gary AT gnu DOT org)