Re: G expressions

[Ludovic Courtès]

Merged!

Here’s the documentation.
Comments and bug reports welcome!

Ludo’.


4.6 G-Expressions
=================

So we have “derivations”, which represent a sequence of build actions to
be performed to produce an item in the store (*note Derivations::).
Those build actions are performed when asking the daemon to actually
build the derivations; they are run by the daemon in a container (*note
Invoking guix-daemon::).

   It should come as no surprise that we like to write those build
actions in Scheme.  When we do that, we end up with two "strata" of
Scheme code(1): the “host code”—code that defines packages, talks to the
daemon, etc.—and the “build code”—code that actually performs build
actions, such as making directories, invoking ‘make’, etc.

   To describe a derivation and its build actions, one typically needs
to embed build code inside host code.  It boils down to manipulating
build code as data, and Scheme’s homoiconicity—code has a direct
representation as data—comes in handy for that.  But we need more than
Scheme’s normal ‘quasiquote’ mechanism to construct build expressions.

   The ‘(guix gexp)’ module implements "G-expressions", a form of
S-expressions adapted to build expressions.  G-expressions, or "gexps",
consist essentially in three syntactic forms: ‘gexp’, ‘ungexp’, and
‘ungexp-splicing’ (or simply: ‘#~’, ‘#$’, and address@hidden), which are
comparable respectively to ‘quasiquote’, ‘unquote’, and
‘unquote-splicing’ (*note ‘quasiquote’: (guile)Expression Syntax.).
However, there are major differences:

   • Gexps are meant to be written to a file and run or manipulated by
     other processes.

   • When a package or derivation is unquoted inside a gexp, the result
     is as if its output file name had been introduced.

   • Gexps carry information about the packages or derivations they
     refer to, and these dependencies are automatically added as inputs
     to the build processes that use them.

   To illustrate the idea, here is an example of a gexp:

     (define build-exp
       #~(begin
           (mkdir #$output)
           (chdir #$output)
           (symlink (string-append #$coreutils "/bin/ls")
                    "list-files")))

   This gexp can be passed to ‘gexp->derivation’; we obtain a derivation
that builds a directory containing exactly one symlink to
‘/gnu/store/…-coreutils-8.22/bin/ls’:

     (gexp->derivation "the-thing" build-exp)

   As one would expect, the ‘"/gnu/store/…-coreutils"’ string is
substituted to the reference to the COREUTILS package in the actual
build code, and COREUTILS is automatically made an input to the
derivation.  Likewise, ‘#$output’ (equivalent to ‘(ungexp output)’) is
replaced by a string containing the derivation’s output directory name.
The syntactic form to construct gexps is summarized below.

 -- Scheme Syntax: #~ EXP
 -- Scheme Syntax: (gexp EXP)
     Return a G-expression containing EXP.  EXP may contain one or more
     of the following forms:

     ‘#$OBJ’
     ‘(ungexp OBJ)’
          Introduce a reference to OBJ.  OBJ may be a package or a
          derivation, in which case the ‘ungexp’ form is replaced by its
          output file name—e.g., ‘"/gnu/store/…-coreutils-8.22’.

          If OBJ is a list, it is traversed and any package or
          derivation references are substituted similarly.

          If OBJ is another gexp, its contents are inserted and its
          dependencies are added to those of the containing gexp.

          If OBJ is another kind of object, it is inserted as is.

     ‘#$PACKAGE-OR-DERIVATION:OUTPUT’
     ‘(ungexp PACKAGE-OR-DERIVATION OUTPUT)’
          This is like the form above, but referring explicitly to the
          OUTPUT of PACKAGE-OR-DERIVATION—this is useful when
          PACKAGE-OR-DERIVATION produces multiple outputs (*note
          Packages with Multiple Outputs::).

     ‘#$output[:OUTPUT]’
     ‘(ungexp output [OUTPUT])’
          Insert a reference to derivation output OUTPUT, or to the main
          output when OUTPUT is omitted.

          This only makes sense for gexps passed to ‘gexp->derivation’.

     address@hidden
     ‘(ungexp-splicing LST)’
          Like the above, but splices the contents of LST inside the
          containing list.

     G-expressions created by ‘gexp’ or ‘#~’ are run-time objects of the
     ‘gexp?’ type (see below.)

 -- Scheme Procedure: gexp? OBJ
     Return ‘#t’ if OBJ is a G-expression.

   G-expressions are meant to be written to disk, either as code
building some derivation, or as plain files in the store.  The monadic
procedures below allow you to do that (*note The Store Monad::, for more
information about monads.)

 -- Monadic Procedure: gexp->derivation NAME EXP [#:system
          (%current-system)] [#:inputs '()] [#:hash #f] [#:hash-algo #f]
          [#:recursive? #f] [#:env-vars '()] [#:modules '()]
          [#:references-graphs #f] [#:local-build? #f]
          [#:guile-for-build #f]
     Return a derivation NAME that runs EXP (a gexp) with
     GUILE-FOR-BUILD (a derivation) on SYSTEM.

     Make MODULES available in the evaluation context of EXP; MODULES is
     a list of names of Guile modules from the current search path to be
     copied in the store, compiled, and made available in the load path
     during the execution of EXP—e.g., ‘((guix build utils) (guix build
     gnu-build-system))’.

     The other arguments are as for ‘derivation’.

 -- Monadic Procedure: gexp->script NAME EXP
     Return an executable script NAME that runs EXP using GUILE with
     MODULES in its search path.

     The example below builds a script that simply invokes the ‘ls’
     command:

          (use-modules (guix gexp) (gnu packages base))

          (gexp->script "list-files"
                        #~(execl (string-append #$coreutils "/bin/ls")
                                 "ls"))

     When “running” it through the store (*note ‘run-with-store’: The
     Store Monad.), we obtain a derivation that procedures an executable
     file ‘/gnu/store/…-list-files’ along these lines:

          #!/gnu/store/…-guile-2.0.11/bin/guile -ds
          !#
          (execl (string-append "/gnu/store/…-coreutils-8.22"/bin/ls")
                 "ls")

 -- Monadic Procedure: gexp->file NAME EXP
     Return a derivation that builds a file NAME containing EXP.

     The resulting file holds references to all the dependencies of EXP
     or a subset thereof.

   Of course, in addition to gexps embedded in “host” code, there are
also modules containing build tools.  To make it clear that they are
meant to be used in the build stratum, these modules are kept in the
‘(guix build …)’ name space.

   ---------- Footnotes ----------

   (1) The term "stratum" in this context was coined by Manuel Serrano
et al. in the context of their work on Hop.