bug#11377: configure.am - fails to remove configure before attempting replacement

[Ronald F. Guilmette]

My apologies for the tardy reply.  I have been off working on other bugs
elsewhere.

In message <address@hidden>, Eric Blake <address@hidden> wrote:

>On 05/03/2012 05:47 PM, Ronald F. Guilmette wrote:
>> In message <address@hidden>, you wrote:
>>..
>>> If things crash in the middle, you can be left without the file,
>>
>> Yes.  So?  You still have the Makefile, which contains all of the 
>> instructions
>> for re-making the file in question.
>>
>>> or with a file with incomplete contents.
>>
>> Could you please explain, for my edification, how inserting an initial `rm'
>> would affect this possibility in any way whatsoever?  (Some examples might
>> help to illustrate.)
>
>See, for example, this autoconf patch:
>http://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=3De9cceec
>
>Autoconf was generating code that created a cache file by writing
>directly in to the target, and there were actual examples of failure in
>the wild where it broke because of a partially written cache file with
>incomplete contents and a current timestamp would not be regenerated to
>be complete.  In fact, the problem was hit in practice so often that gcc
>had resorted to a per-directory config.cache just to work around the
>worst aspects of this failure mode.

OK, so sometimes Makefile target files can get only partially (and
inadequattely) written.  Granted.

However you still haven't demonstrated how an initial `rm' would have
made any difference at all to that specific problem, one way other
the other.  (It doesn't make the problem worse and it doesn't make
the problem better either.  It has no effect at all on the problem
you cite.)  Neither have you provided any example in which an initial
`rm' at the start of the commands recipie to rebuild the target file
would in fact have any deliterious effects whatsoever.

In short, your assertion that doing an initial `rm' would be somehow a
Bad Thing, on account of the potential ``partially written targets''
file problem simply does not stand up to scrutiny.  This potential for
partially-written target files exists and continues to exist entirely
independently of whether or not the rules to rebuild any given target
file include an initial `rm' of that target file.

>We switched things over to writing to a temporary file and atomically
>mv'ing it into place, and the bug disappeared.

That's swell, and that that is certainly one way of eliminating the
your specific ``partially written targets'' issue.  (I can think of a
few other obvious solutions, including creating a .stamp file after
your actual target file has been completely updated... the way Stallman
was doing in the GCC sources, 20+ years ago.)

The important point remains however:  My suggestion for doing an
initial `rm' on certain select files created as a consequence of using
automake has no bearing on, and is entirely irrelevant and orthogonal
to the ``partially written targets'' problem... and vise versa.  Doing
(or not doing) an initial `rm' for a Makefile target file neither creates
that problem, nor solves that problem.  As I say, it is entirely
irrelevant and orthogonal to that entirely different problem.

>> I'm not even sure how one would generate a file with "incomplete contents",

Please excuse me.  That was a downright stupid thing for me to have said.
Of course I know how a partially-written target file might come about,
and I also have seen such things, on occasions, in practice.  Really, I
think that I was just trying to elicit real examples from you in order
to allow me some concrete basis to make my point, i.e. that an initial
`rm' of a Makefile target file is _never_ harmful... well... except in
those few and extremely rare cases associated with use of the .PRECIOUS
pseudo-target.  (But those are the exceptions that prove the rule... an
initial `rm -f' in the recipies for almost all Makefile targets is, in
general and on balance, a Good Thing, leading to higher reliability of
Makefiles in general.)

>> In any event, the style of Makefile coding you are suggesting, wherein
>> the final command to remake a given target would be something like:
>>
>>      mv tempfile target
>>
>> is not, I would suggest, at all widespread in current practice.
>
>Agreed, but that doesn't mean that we shouldn't advertise it as being
>safer than rm.

See above.  The two possibilities are separate and orthogonal.

Any given target file may or may not be subjected to an initial `rm -f'
at the start of the recipie to rebuild it.  (And as I have tried to point
out, in general it is Better to including such an initial `rm -f' in
essentially _every_ case I can think of.)

Separately and independently, some subset of all Makefile target files
are generated in such a way that they are prone to being only partially
re-written, e.g. if make is interrupted by cntl-C.  (Does this in-
clude even essentially all .o files??  One would think so.)  This is a
separate and unrelated problem.  In some few cases it becomes bad enough
to pay attention to (and to do something about) e.g. as in your autoconf
cache file example.  In those cases, one obvious solution... although
certainly not the only one... is to generate the desired file contents
first into a temporary file and then... ``atomically''... perform a
`mv' to move the completely built file contents to the actual desired
target file name.

These two possibilities are not in any way conflicting.  One can write
the recipie to rebuild a given target such that it contains an initial
`rm' (or not) and also so that it contains a final `mv' (or not).  These
possibilities are not mutually exclusive.

>> P.S.  Your suggestion that removing & replacing the contents of a given 
>> target
>> file only at the very last second (and only in an atomic operation) is in 
>> some
>> sense "best practice" assumes that something would, in fact, go wrong if
>> the given target file were to be removed sooner, before being replaced.=
>
>Yes - in the case of autoconf's config.cache, things DO go wrong in a
>subsequent build if config.cache is removed too soon (a partial file can
>corrupt the build,

See above.  To the best of my knowledge, the `rm' command is _never_
responsible for creating partially/incompletely written target files.

> and a missing file costs extra time to rebuild the file).

Oh come come now!  You are seriously grasping at straws to make your point
now.

And in fact, your (already weak) argument that doing an `rm' too soon on
a given target file would incur some (tiny?) subsequent processing time
penalty is provably just plain wrong.  Consider a simple example:


        foo: bar
                -rm -f foo
                ...

Your claim is that execution of the initial `rm -f' of foo might result 
in foo having to be rebuilt at some later time (e.g. if the make is
interrupted after the `rm -f' and then restrarted later on) and that
this creates some unnecessary processing overhead.

But consider this... If in fact that `rm -f' line had indeed been executed
then isn't it the case that that could ONLY have occured because make,
together with the rules in the Makefile, together with the current state
of the local environment had _already_ determined that foo needed to be
rebuilt anyway?

In other words, you seem to be arguing that the initial `rm -f' might
possibly result in some processing overhead, in certain circumstances
(e.g. ^C) to rebuild `foo', but that argument does not seem to hold water
because it was _already_ the case that foo needed to be rebuilt anyway.
Thus these is no extra or additional overhead created by the presence of
that initial `rm -f'.


Regards,
rfg