New ‘guix offload’ build hook

[Ludovic Courtès]

Hi!

Finally support for offloading builds to other Guix machines has landed!
It’s still rough on the edges, but it does the job, and should allow us
to start having more than one machine (and one architecture) building
things.  Yay!

It requires Guile 2.0.10 (not yet released) though, and is not built
when this prerequisite is missing.

Below is the new documentation, for comments.  The design is obviously
similar to that of Nix’s remote build facility, but I think it’s also
more flexible (machine file, PKI, etc.)

Future work:

  • Currently it connects to remote machine using GNU lsh, but I’d like
    to switch to Guile-SSH, because that will provide finer control.

  • On IRC Mark rightfully pointed out that it would be nice to keep a
    signature of the machine that actually built an element (currently,
    signatures are checked and then discarded.)  I realized they could
    actually easily be kept around in $localstatedir, and possibly
    reused as-is upon export.

  • Fixing bugs, FIXMEs, etc.

Ludo’.


2.2.2 Using the Offload Facility
--------------------------------

The build daemon can "offload" derivation builds to other machines
running Guix, using the ‘offload’ "build hook".  When that feature is
enabled, a list of user-specified build machines is read from
‘/etc/guix/machines.scm’; anytime a build is requested, for instance via
‘guix build’, the daemon attempts to offload it to one of the machines
that satisfies the derivation’s constraints, in particular its system
type—e.g., ‘x86_64-linux’.  Missing prerequisites for the build are
copied over SSH to the target machine, which then proceeds with the
build; upon success the output(s) of the build are copied back to the
initial machine.

   The ‘/etc/guix/machines.scm’ is—not surprisingly!—a Scheme file whose
return value must be a list of ‘build-machine’ objects.  In practice, it
typically looks like this:

     (list (build-machine
             (name "eightysix.example.org")
             (system "x86_64-linux")
             (user "bob")
             (speed 2.))    ; incredibly fast!

           (build-machine
             (name "meeps.example.org")
             (system "mips64el-linux")
             (user "alice")
             (private-key
              (string-append (getenv "HOME")
                             "/.ssh/id-rsa-for-guix"))))

In the example above we specify a list of two build machines, one for
the ‘x86_64’ architecture and one for the ‘mips64el’ architecture.  The
compulsory fields for a ‘build-machine’ declaration are:

‘name’
     The remote machine’s host name.

‘system’
     The remote machine’s system type.

‘user’
     The user account to use when connecting to the remote machine over
     SSH. Note that the SSH key pair must _not_ be passphrase-protected,
     to allow non-interactive logins.

A number of optional fields may be optionally specified:

‘private-key’
     The SSH private key file to use when connecting to the machine.

‘parallel-builds’
     The number of builds that may run in parallel on the machine (1 by
     default.)

‘speed’
     A “relative speed factor”.  The offload scheduler will tend to
     prefer machines with a higher speed factor.

‘features’
     A list of strings denoting specific features supported by the
     machine.  An example is ‘"kvm"’ for machines that have the KVM
     Linux modules and corresponding hardware support.  Derivations can
     request features by name, and they will be scheduled on matching
     build machines.

   The ‘guix’ command must be in the search path on the build machines,
since offloading works by invoking the ‘guix archive’ and ‘guix build’
commands.

   There’s one last thing to do once ‘machines.scm’ is in place.  As
explained above, when offloading, files are transferred back and forth
between the machine stores.  For this to work, you need to generate a
key pair to allow the daemon to export signed archives of files from the
store (*note Invoking guix archive::):

     # guix archive --generate-key

Thus, when receiving files, a machine’s build daemon can make sure they
are genuine, have not been tampered with, and that they are signed by an
authorized key.