Re: GSoC: OOD detection

[Paul Smith]

On Mon, 2007-04-16 at 23:15 +0200, Ramón García wrote:
> I am not sure if I get your method of using persistent variables. Each
> target has a variable associated with it. If a file is found not out
> of date, the persistent variable associated with it is empty. Is this
> empty value stored?

Sorry, I wasn't clear about this.  No, the value is not stored.  If the
target doesn't need to be rebuilt then we don't change the persistent
state.


> Out of date determination must be determined bottom to top, because
> "a" may be up to date now, but after "a1" is rebuilt it may be no
> longer up to date. If "a1" is updated, its persistent value is also
> updated. Suppose that this value is the md5sum.

Obviously it is never sufficient to store the md5sum of the TARGET as
the persistent state; that's relatively uninteresting to make.  The
persistent state for a given target with an md5sum-based OOD algorithm
must be the md5sum's of all the PREREQUISITES.  The target is irrelevant
(unless you want to be super-careful; see below).  Here's an example:

        A: B C D

What make wants to know, really, is one of two things: first, does A
exist?  If not, then regardless of the status of B, C, and D we must
rebuild A.  So this case is not interesting (but worth noting, since it
still behaves correctly if A has never been built and thus has no
persistent state from a previous build).

If A does exist, then make wants to know whether any of B or C or D have
changed since the last time A was updated.  When we use
time-last-modified as an OOD we compare A's TLM to those of B, C, and D
to figure this out.

But, checking the status of the file A is not required if we can know
that B, C, or D were updated in some other way.  If we have persistent
state, we CAN know that, because the md5sum of one or more of B, C, or D
will be different than they used to be.  In other words, we're comparing
the current state of B with its previous state when A was built, and
similar for C and D.  We're not comparing A, itself, at all.

So, a pure md5sum expansion of the .OUT_OF_DATE variable for target A
might look something like:

        B:<md5sum-of-B> C:<md5sum-of-C> D:<md5sum-of-D>

or whatever is easy for make to parse.  The next time make runs it
computes the md5sum for file B and compares it to the one in the
persistent state for A, etc.

Of course, some people want to be even more careful and they DO want to
add the md5sum for A as well, so that if someone or something modifies A
outside of make's knowledge, make can know it should be rebuilt even if
B, C, and D are the same.  Other options, such as storing the command
script to see if it's changed, are in the same boat.  As I've mentioned
before, these cannot be handled by simply storing the result of the OOD
computation, since that happens BEFORE the target is built.


Obviously there are significant performance considerations, but I think
we should put these aside for now, and examine them AFTER the user
interface is defined.  I expect we can manage the performance issues
with some well-placed builtin functions, or plugins as David suggested,
or something.

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <address@hidden>          Find some GNU make tips at:
 http://www.gnu.org                      http://make.paulandlesley.org
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist