Fwd: What is the logic behind make's handling of symbolic links?

[Philip Couling]

Thanks both lots for your responses.  Its always interesting to hear these
views.

This behavior still seems wrong to me.  I've hit a scenario where this
behavior is distinctly unhelpful and supprising
<http://en.wikipedia.org/wiki/Principle_of_least_astonishment>. I'm
struggling to find a scenario where this is both helpful and intuitive.

The most simple demonstration of this surprise is that if I have these
rules:
X:| Y
    foo Y X

Y:
    bar Y

then deleting  the file Y and then running make would not intuitively force
the rebuild of X.  But if I tell you that foo creates a symbolic link then
suddenly the behavior changes, and removing Y *erroneously* forces the
rebuild build of X.  Thus the syntax of make is not providing an intuitive.

I disagree with the idea that a dangling symlink "does not exist".  This is
an oversimplification and glosses over the distinction between a file's
name and a file's content.  I've not see an operating system which doesn't
recognize this distinction since DOS .  More accurately then; a dangling
symlink can be thought of as a file name which does exist but who's content
is missing.  Most programs appear to follow this model rather than a more
binary "both exist or neither".  Even cp's error message acknowledges the
existence of the dangling link (though it does refuse to write to it).

Make is NOT doing the same thing as cat...
Cat is attempting to read content - presumably using open (man 2 open
<http://linux.die.net/man/2/open>).
make is checking file-name existence (and timestamps) - presumably using
stat (man 2 stat <http://linux.die.net/man/2/stat>)

make is fundamentally interested in file names.  In no other context is
make interested in file content.  This makes it extremely surprising when
make takes an interest in content for this one and only purpose
(symlinks).  It would be less surprising if it worked solely with file
names full stop.  The technical difference here is as simple as using lstat
(man 2 lstat <http://linux.die.net/man/2/lstat>) instead of stat.

As already stated, the current design does trip up developers and does
force work arounds (such as those given by Britton).  I'm looking for a
scenario where the result of this would be surprising to a developer.  I'm
looking for a scenario where this would force a work around from a
developer.

Are you aware of any such examples?





On 21 October 2014 18:00, Etan Reisner <address@hidden> wrote:

make is doing the same thing cat is doing here I believe. Trying to
> dereference the symlink, failing to find a valid entry on the other
> end and giving up. Note that the error from cat is not "dangling
> symlink" it is "No such file or directory". That's indistinguishable
> from the symlink not existing. Note also that trying to write through
> a dangling symlink  isn't doing anything special here either. If the
> application expects the target to exist and it doesn't it will fail,
> otherwise it creates the file but this is still "transparent" in terms
> of the symlink itself existing.
>
> Given that make is operating in a perfectly reasonable way, the file
> doesn't exist. When files don't exist make tries to build them. What
> you are asking for is for make to explicitly add the concept of a
> symlink to its understanding of file system semantics (and then you
> would need to be able to control this behaviour, like the -L/-H/-P
> flags to find, etc.) to ensure you get the behaviour you want from
> make. This seems unnecessarily complicated to me.
>
> Listing the contents of a directory is a different case (and make's
> $(wildcard) function should find them that way as well) since it is
> walking directory entries from the directory itself and not dealing
> with the "contents" of the file entry in any way.
>
>     -Etan


On 21 October 2014 19:18, Britton Kerin <address@hidden> wrote:

> On Tue, Oct 21, 2014 at 5:00 AM, Philip Couling <address@hidden> wrote:
> > Hi
> > I was wondering why make treats dangling symbolic links as if they don't
> > exist?
>
> This is consistent behavior (see below)
>
> > I've been trying to create a set of links as targets.  The problem I have
> > is that make's handling of dangling pointers makes this unfortunately
> > complicated.
> >
> > It seems that a decision has been taken to treat dangling links as if
> they
> > don't exist.  This means that if the referenced file does not exist at
> the
> > start of the build process then make will try to re-create the link even
> if
> > the build process will create the referenced file.  This results in
> errors
>
> If the build process is going to create the referenced file, perhaps you
> should make a target of the symlink?  You can include the real file in
> the rule as well if you want:
>
> mylinktarget mylink:
>         echo stuff_to_go_in_mylinktarget >mylinktarget
>
> Depending on side effects which aren't in make's DAG for file generation
> often results in confusion.
>
> > because ln will not overwrite the link.
>
> ln --force might work for you
>
> > Symlinks are a conceptually strange phenomenon and it's clear that
> handling
> > them is very application specific:
> >
> >    - Programs which try to read a link's content will fail if the
> >    referenced file doesn't exist. (Try cat mylink)
>
> True, and this is how make behaves as well.
>
> >    - Programs' which try to write to a link's content will succeed by
> >    creating the referenced file if possible.  (Try cp myfile mylink )
>
> Not true:
>
>   $ touch real_file
>   $ ln -s nonexistent broken_symlink
>   $ cp real_file broken_symlink
>   cp: not writing through dangling symlink `broken_symlink'
>   1 $
>
> >    - Programs which list a directory or check for existence will succeed
> >    and see them as existing. (ls mylink )
>
> Not true in general:
>
>   $ ln -s nonexistent broken_symlink
>   $ test -e broken_symlink || echo broken_symlink does not exist
>   broken_symlink does not exist
>
> > So the way that make interprets dangling links seems less standard and
> has
> > an identifiable problem.
>
> Make seems right to me.  The general rule is to treat symlinks as
> transparent
> except where doing so looks both wrong and destructive, and make follows
> this policy.  It is true that having broken symlinks in your sources tends
> to cause trouble.  I usually avoid it, except when its really tempting and
> easy :)  I stick things like this in my Makefiles:
>
> # Regard most broken symbolic links as errors {{{1
>
> # Broken Symbolic Link Exception Patterns
> BSLEP = ./make_part_data/%
>
> BROKEN_SYMBOLIC_LINKS = $(filter-out $(BSLEP),$(shell find -L . -type l))
>
> ifneq ($(strip $(BROKEN_SYMBOLIC_LINKS)),)
>   $(error the following broken symbolic links exist:
> $(BROKEN_SYMBOLIC_LINKS))
> endif
>
> # }}}1
>
> Britton
>