Re: [patch] sort 'find' output to enable deterministic builds.

[Ralf Wildenhues]

* Chris Demetriou wrote on Tue, Mar 16, 2010 at 08:28:12AM CET:
> On Mon, Mar 15, 2010 at 23:37, Ralf Wildenhues wrote:
> > libtool also uses 'ar' in a number of places and cases, most
> > prominently, but not limited to, the static linking scenario.
> > Do you have measures in place to use 'D' there too?
> 
> Yes.  At least in all the places I've noticed (or that are relevant
> for a Linux / ELF build), libtool actually uses $AR.  Likewise with
> $RANLIB.
> 
> If you want a repeatable build that avoids ar timestamps, you need:
>   AR=... wrapper script that adds 'D' and strips 'u' from ar's first arg ...
>   RANLIB=/bin/true or wrapper script that invokes /bin/true.

OK.

> (i didn't grep through all of the libtool code, but AFAICT at least
> ltmain.sh uses $AR and $RANLIB in preference to hard-coding ar and
> ranlib ... which makes sense, since, for example, it's gotta be
> cross-compilation friendly.)

Yes.

> > IIRC some (non-GNU) compilers (or the assemblers they call, I don't
> > recall) also add time stamps to generated objects.  Are they relevant to
> > you?  I'm not sure whether one can turn stamps off everywhere; have you
> > looked into normalizing output like in GCC's contrib/compare-debug?
> 
> In my experience, it's the object formats.

Yes, that may be it.  However, that also means that, while the patch
fixes things for you, it doesn't really add value to Libtool in the
sense that we cannot guarantee an improvement of some portable kind to
users.  That's ok per se, but of course not ideal.  ;-)

> E.g., COFF objects
> (including ECOFF and PE) include a timestamp.  Looks like BFD puts in
> 0 for that in most cases, but e.g. the bfd/peXXigen.c code looks like
> it *will* generate a timestamp.  Those object file formats are not
> relevant to me, though i've seen environments before were they're
> overridden.  (I once wrote a small set of tools to do a
> timestamp-insensitive compare for windows builds.  I've thankfully
> been *mostly* COFF-free for the past 10+ years.  8-)
> 
> (BTW, my purpose isn't to be able to compare output files so much as
> create bit-identical output files.  There are several reasons for
> this, but in summary, comparison isn't quite good enough.

Wanna share a couple of those reasons why comparison isn't good enough?

>  In order to
> do this, you've gotta do a bunch of other things, e.g., build with the
> same paths, use -frandom-seed appropriately if building some C++ code,
> ...)

> > Then, it would be nice to be able to confirm that we produce stable
> > output in cases where this is desirable; IOW, have testsuite exposure.
> > Can you describe your setup in a bit more detail?  (You could also
> > provide a test case, but that will probably require copyright papers
> > then first.)
> 
> Actually,  my employer has a blanket assignment AFAIU, so that's not a 
> problem.

Wasn't aware of that.

Thanks,
Ralf