Re: [bootstrappable] diverse double compilation: using $ORIGIN?

[Jan Nieuwenhuizen]

Ricardo Wurmus writes:

> When building packages, Guix embeds the absolute file name of the output
> directory in the resulting binary.  That’s usually fine and what we
> want.
>
> Now let’s assume that we’d like to build the GCC sources with diverse
> double compilation: we’d have a “gcc” package that’s built with
> “gcc-core”; we’d also have a “gcc-from-clang” package that uses Clang as
> its input.  Since the GCC build procedure is performed at least two
> times (once with the bootstrap compiler, and then again with the GCC
> variant this produces), the resulting GCC binaries should be identical.
>
> Except that they are not.  One of the reasons is that the binaries that
> Guix produces embed the target output directories.  This means that the
> two compiler binaries that result from diverse double compilation will
> *always* differ in at least the embedded paths, such as paths to itself
> (e.g. to binaries in the libexec directory) and paths to.
>
> I wonder if we can use $ORIGIN in the compiler binaries, so that we can
> avoid (most) references to the output directories, thereby making the
> equality check between the diversely built compiler binaries simpler.
>
> Is this at all possible?  Or do we need to accept that the equality
> check for diverse double compilation for binaries built with Guix must
> be aware of /gnu/store references and ignore those?
>
> Alternatively, could we move all of these store references into a
> wrapper script that would tell the binary about them via environment
> variables?  The equality check would only need to exclude the wrapper
> script then and compare the two sets of compiler binaries naively.
>
> Does this make any sense?

Attached is a patch prototyping this diverse double compilation test.

janneke

>From c91609e847066c384826d726033146e08d8185ed Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <address@hidden>
Date: Thu, 2 Nov 2017 06:52:46 +0100
Subject: [PATCH] gnu: Add clang-gcc, gcc-ddc.  WIP

Usage: guix build gcc-dcc

Building gcc-dcc tests the diverse double compilation property
of the gcc that Guix is using.

gcc-dcc is a meta-package that depends on gcc-5.4.0 and on
clang-gcc-5.4.0 (the same GCC built with clang).  The builder
checks if both gcc's are bit-for-bit identical and fails if
they differ.

* gnu/packages/bootstrappable.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
---
 gnu/local.mk                    |  1 +
 gnu/packages/bootstrappable.scm | 98 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 99 insertions(+)
 create mode 100644 gnu/packages/bootstrappable.scm

diff --git a/gnu/local.mk b/gnu/local.mk
index 90dc7aec1..379b2c7b9 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -78,6 +78,7 @@ GNU_SYSTEM_MODULES =                          \
   %D%/packages/boost.scm                       \
   %D%/packages/bootloaders.scm                 \
   %D%/packages/bootstrap.scm                   \
+  %D%/packages/bootstrappable.scm              \
   %D%/packages/build-tools.scm                 \
   %D%/packages/busybox.scm                     \
   %D%/packages/c.scm                           \
diff --git a/gnu/packages/bootstrappable.scm b/gnu/packages/bootstrappable.scm
new file mode 100644
index 000000000..4ccde7a6d
--- /dev/null
+++ b/gnu/packages/bootstrappable.scm
@@ -0,0 +1,98 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2017 Jan Nieuwenhuizen <address@hidden>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages bootstrappable)
+  #:use-module ((guix licenses)
+                #:select (gpl3+))
+  #:use-module (gnu packages)
+  #:use-module (gnu packages base) ; diffutils
+  #:use-module (gnu packages gcc)
+  #:use-module (gnu packages llvm)
+  #:use-module (gnu packages package-management) ; diffosscope
+  #:use-module (guix packages)
+
+  #:use-module (guix build-system trivial)
+
+  ;; for clang-gcc
+  #:use-module (guix build-system gnu)
+  #:use-module (guix utils)
+  #:use-module (gnu packages bootstrap)
+  #:use-module (gnu packages compression)
+  #:use-module (gnu packages elf)
+  #:use-module (gnu packages multiprecision)
+  #:use-module (gnu packages texinfo)
+
+  #:use-module (srfi srfi-1))
+
+(define gcc-configure-flags-for-triplet (@@ (gnu packages gcc) 
gcc-configure-flags-for-triplet))
+
+(define-public clang-gcc
+  (package
+    (inherit gcc-5)
+    ;;(name "clang-gcc")
+    (name "cgc")                        ; let's see if length matters here
+    (version "5.4.0")
+    (native-inputs `(("clang" ,clang)
+                     ,@(alist-delete "gcc" (package-native-inputs gcc-5)))) ; 
FIXME: %final-inputs?
+    (arguments
+     (substitute-keyword-arguments (package-arguments gcc-5)
+       ((#:phases original-phases)
+        `(modify-phases ,original-phases
+           (add-before 'configure 'setenv
+             (lambda* (#:key inputs #:allow-other-keys)
+               (let ((clang (assoc-ref inputs "clang")))
+                 (setenv "CC" (string-append clang "/bin/clang"))
+                 (setenv "CXX "(string-append clang "/bin/clang++")))))))))))
+
+(define-public gcc-ddc-gcc+clang
+  (package
+    (name "gcc-ddc")
+    (version "5.4.0")
+    (source #f)
+    (native-inputs `(("diffoscope" ,diffoscope)
+                     ("diffutils" ,diffutils)
+                     ("gcc" ,gcc-5)
+                     ("clang-gcc" ,clang-gcc)))
+    (build-system trivial-build-system)
+    (arguments
+     `(#:modules ((guix build utils))
+       #:builder
+       (begin
+         (use-modules (guix build utils))
+         (let* ((diffoscope (assoc-ref %build-inputs "diffoscope"))
+                (diffutils (assoc-ref %build-inputs "diffutils"))
+                (gcc (assoc-ref %build-inputs "gcc"))
+                (gcc/bin/gcc (string-append gcc "/bin/gcc"))
+                (clang-gcc (assoc-ref %build-inputs "clang-gcc"))
+                (clang-gcc/bin/gcc (string-append clang-gcc "/bin/gcc")))
+           ;; FIXME: diffoscope.exc.RequiredToolNotFound: cmp
+           (setenv "PATH" (string-append diffoscope "/bin:"
+                                         diffutils "/bin:"))
+           ;; diffoscope.presenters.formats: Console is unable to print 
Unicode characters. Set e.g. PYTHONIOENCODING=utf-8
+           (setenv "PYTHONIOENCODING" "utf-8")
+           ;; cmp should suffice but gives little insight if it fails
+           ;; (zero? (system* "cmp" gcc/bin/gcc clang-gcc/bin/gcc))
+           ;; for starters, only check the gcc binary
+           (zero? (system* "diffoscope" gcc/bin/gcc clang-gcc/bin/gcc))
+           ))))
+    (synopsis "test gcc+clang DDC property for gcc-5.4.0")
+    (description "gcc-dcc is a meta-package that depends on gcc-5.4.0 and on
+clang-gcc-5.4.0 (the same GCC built with clang).  The builder checks if both
+gcc's are bit-for-bit identical and fails if they differ.")
+    (home-page "http://bootstrappable.org";)
+    (license gpl3+)))
-- 
Jan Nieuwenhuizen <address@hidden> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

-- 
Jan Nieuwenhuizen <address@hidden> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com