PATCHES: Brain surgery on the build system, first stage

[John Mandereau]

Hi guys,

I tried to submit commits in dev/jmandereau not in master to Rietveld
(git cl upload origin/master), but git-cl said just after I edited the
issue description

No output from ['git', 'show', 'aclocal.m4']

and exited.  So below goes the issue description, that includes
instructions to test the changes (even if I could upload the issue to
Rietveld, Patchy would not suffice anyway).  I kindly ask a review of
the four commits on top of master that are in dev/jmandereau branch on
Savannah git repo.

Best,
John

==============================================
Brain surgery on the build system, first stage

Sorry to do this just before 2.16 branching out, but if you remember
nightmares about non-distributed files and the like because of the
desire to include last-minute changes to translations and whatnot, IMO
the part of this patch regarding "make dist" is really worth the
trouble.  If you like, I can also include a merge of stepmake/stepmake
and make/, but I thought this would really clutter the review of other
changes, and this merge is trivial if you compare with the patch of
this issue, so that merge will be submitted for review after this
review issue settled down.

Commits are viewable separately on dev/jmandereau:
http://git.savannah.gnu.org/gitweb/?p=lilypond.git;a=shortlog;h=refs/heads/dev/jmandereau
I recommend inspecting changes locally with git with "git log -p -M"
to sort out real changes in moved files.

Patchy test is welcome, but is not enough.  People who have reasonable
horsepower (or many hours of CPU time to allocate for this, typically
12 hours on a Pentium 4 with 2GB RAM) and about five gigabytes of free
space might want to run the following test.

I checked which files disappeared or appeared with these changes in
both tarball and installation of binary and docs, with the following
"scripts" (they are very dirty, but I'm not sure it's worth adding
installation and distribution checks to Patchy), assuming you've set
LILYPOND_GIT and LILYPOND_BUILD_DIR as in
http://lilypond.org/doc/v2.15/Documentation/contributor/environment-variables
and you start from master branch checked out in $LILYPOND_GIT:

export VERSION=2.15.43
export MAKE_FLAGS=-j3 CPU_COUNT=3
./autogen.sh --noconfigure && \
rm -rf $LILYPOND_BUILD_DIR && \
mkdir -p $LILYPOND_BUILD_DIR && \
cd $LILYPOND_BUILD_DIR && \
$LILYPOND_GIT/configure && \
mkdir -p $HOME/lily-base && \
make dist &>$HOME/lily-base/make-dist.log && \
mv out/lilypond-${VERSION}.tar.gz $HOME/lily-base && \
pushd $HOME/lily-base && tar -xzf lilypond-${VERSION}.tar.gz && popd && \
make distclean && \
$LILYPOND_GIT/configure --prefix=$HOME/lily-base && \
make ${MAKE_FLAGS} &>$HOME/lily-base/make.log && \
make ${MAKE_FLAGS} test &>$HOME/lily-base/make-test.log && \
make ${MAKE_FLAGS} doc &>$HOME/lily-base/make-doc.log && \
make install &>$HOME/lily-base/make-install.log && \
make install-doc &>$HOME/lily-base/make-install-doc.log

Cd to $LILYPOND_GIT, apply patch (or checkout dev/jmandereau), then do

./autogen.sh && \
make vc-clean && \ 
./autogen.sh --noconfigure && \
rm -rf $LILYPOND_BUILD_DIR && \
mkdir -p $LILYPOND_BUILD_DIR && \
cd $LILYPOND_BUILD_DIR && \
$LILYPOND_GIT/configure && \
mkdir -p $HOME/lily-new && \
make dist &>$HOME/lily-new/make-dist.log && \
mv out/lilypond-${VERSION}.tar.gz $HOME/lily-new && \
pushd $HOME/lily-new && tar -xzf lilypond-${VERSION}.tar.gz && popd && \
make distclean && \
$LILYPOND_GIT/configure --prefix=$HOME/lily-new && \
make ${MAKE_FLAGS} &>$HOME/lily-new/make.log && \
make ${MAKE_FLAGS} test &>$HOME/lily-new/make-test.log && \
make ${MAKE_FLAGS} doc &>$HOME/lily-new/make-doc.log && \
make install &>$HOME/lily-new/make-install.log && \
make install-doc &>$HOME/lily-new/make-install-doc.log

Cd to $HOME, check the output of
LANG= diff -qr lily-base lily-new |grep -i only
then test in-tree build (first line is optional, it's merely
here to free some disk space):

rm -rf $LILYPOND_BUILD_DIR && \
cd $LILYPOND_GIT && \
./configure && \
mkdir -p $HOME/lily-new-intree && \
make dist &>$HOME/lily-new-intree/make-dist.log && \
mv out/lilypond-${VERSION}.tar.gz $HOME/lily-new-intree && \
pushd $HOME/lily-new-intree && tar -xzf lilypond-${VERSION}.tar.gz && popd && \
make distclean && \
./configure --prefix=$HOME/lily-new-intree && \
make ${MAKE_FLAGS} &>$HOME/lily-new-intree/make.log && \
make ${MAKE_FLAGS} test &>$HOME/lily-new-intree/make-test.log && \
make ${MAKE_FLAGS} doc &>$HOME/lily-new-intree/make-doc.log && \
make install &>$HOME/lily-new-intree/make-install.log && \
make install-doc &>$HOME/lily-new-intree/make-install-doc.log

Again, cd to $HOME and see the output of
LANG= diff -qr lily-base lily-new-intree |grep -i only

You should in either case (lily-new, lily-new-intree) see differences
that match either deleted or moved files in the tarball, or
lilypond-book hashing.


commit 61d98da98ea74c5fdb648ea69a400feb8959b37e
Improve make *clean targets: rename cvs- to vc-, adjust distclean

* distclean was usable from any directory in theory, make it toplevel-only,
* vc-clean (was cvs-clean): trigger doc-clean then distclean in addition,
* remove maintainerclean, which was a broken distclean-like.


commit 6e41aa23213fc8dc4efa58bee4adbe5f1cf62613
Remove StepMake as a separate package

Rationale:
* there are no publicly-known current usage cases of StepMake outside
LilyPond,
* StepMake installation has been disabled for a decade,
* generic templates (stepmake/stepmake) have been so heavily
customized lately for our heavy documentation that they can no longer
be qualifed as "generic",
* only a few build scripts have remained in stepmake/bin, whereas
most of them now live in scripts/build.

This commit removes everything in stepmake/ except included makefiles;
scripts that are still used are moved to config/ (Autoconf helpers) or
scripts/build (build and install scripts).

Remove config make target to avoid potential conflict with new
config/ directory.

For clarity, a merge of all stepmake templates, from stepmake/stepmake
and make/ into stepmake/ (or similar), will be done in another commit.


commit 8a0905f1409f3d01bf902312cb53cb8f1127e760
Make distributed tarball from Git file list

This removes the requirement of having a GNUmakefile
in each and every directory of the source tree; this commit
also deletes GNUmakefiles made unnecessary this way.

dist-toplevel-txt-files toplevel target is also removed,
because it is unneeded outside of dist target and it
prevents linking of toplevel generated txt docs to
distdir when these docs are already generated.

commit 79accfcc11f725cb08951567b8da80eaf2183517
mass-link: handle filesystem crossing and support OS without link support