Re: Using cuirass to build your own manifest.

[ng0]

Mathieu Othacehe transcribed 5.9K bytes:
> 
> Hi,
> 
> Here's a small tutorial on how to setup cuirass to build your own
> manifest.

Nice :) I am working on a similar documentation, it would just extend
this by providing examples of how to do this via .onion, via OpenNIC dns
name, and inside the freifunk meshnetwork.
If I mention you as the original author may I just extend the text, and
add my changes?

> I see two major reasons for this kind of setup:
> 
> * When you pull latest guix, hydra and bayfront may not have finished
>   building all the packages you use.
> 
> * Hydra and bayfront won't build your custom packages.
> 
> For this reasons I installed cuirass to build my manifest, using a
> configuration very similar to the one used on bayfront
> (http://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/bayfront.scm).
> 
> I guess it may not be the easier way to do it so don't hesiste to
> comment. Anyway, here's how I set it up.
> 
> 1) config.scm
> 
> --8<---------------cut here---------------start------------->8---
> (define %cuirass-specs
>   ;; Cuirass specifications to build Guix.
>   #~(list `((#:name . "guix")
>             (#:url . "git://git.savannah.gnu.org/guix.git")
>             (#:branch . "master")
>             (#:no-compile? #t)
>             (#:load-path . ".")
>             ;; Eval function drv-list in file guix-drv.scm
>             (#:proc . drv-list)
>             (#:file . #$(local-file 
> "/home/mathieu/conf/guix/cuirass/guix-drv.scm")))))
> (services
>  ...
>  (service cuirass-service-type
>           (cuirass-configuration
>            (interval 30) ;; git pull guix repo every 30 seconds.
>            (use-substitutes? #t)
>            (port 8082)
>            (load-path '("/home/mathieu/conf/guix/packages"
>                         "/home/mathieu/conf/guix/common_packages"))
>            (specifications %cuirass-specs))))
> --8<---------------cut here---------------end--------------->8---
> 
> With this configuration, cuirass service will wake-up every 30
> seconds. It will pull branch master of guix.git. If new commits have
> appeared since last evaluation, a build will be triggered.
> 
> Cuirass will evaluate the function specified with #:proc in the file
> #:file. This function is supposed to return the list of derivations you
> want to build.
> 
> Note that if this list contains custom packages, you have to setup
> load-path to point to the directories containing the custom packages
> definitions.
> 
> 2) guix-drv.scm
> 
> --8<---------------cut here---------------start------------->8---
> (use-modules (guix config)
>              (guix store)
>              (guix grafts)
>              (guix packages)
>              (guix ui)
>              (guix derivations)
>              (guix monads)
>              (guix profiles)
>              (gnu packages)
>              (srfi srfi-1))
> 
> (define (drv-package store package)
>   (lambda ()
>     `((#:job-name . ,(string-append
>                       (package-name package)
>                       "-"
>                       (package-version package)
>                       "-job"))
>       (#:derivation . ,(derivation-file-name
>                         (parameterize ((%graft? #f))
>                           (package-derivation store package #:graft? #f)))))))
> 
> (define (drv-list store arguments)
>   (let* ((manifest
>          (load* "/home/mathieu/conf/guix/manifest.scm"
>                 (make-user-module
>                  '((guix profiles) (gnu)))))
>          (packages
>           (map manifest-entry-item
>                (manifest-entries manifest))))
>     (parameterize ((%graft? #f))
>       (map (lambda (package)
>              (drv-package store package))
>            (delete-duplicates! packages)))))
> --8<---------------cut here---------------end--------------->8---
> 
> The drv-list procedure loads the file manifest.scm which content is
> detailed below. The list produced by drv-list looks like :
> 
> --8<---------------cut here---------------start------------->8---
> (((#:job-name . "acpi-1.7-job") (#:derivation
> . "/gnu/store/r9s5x0ksj02hsw4n3acdxab8ggjp4z7y-acpi-1.7.drv")) ...)
> --8<---------------cut here---------------end--------------->8---
> 
> 3) manifest.scm
> 
> --8<---------------cut here---------------start------------->8---
> (define (spec->packages spec)
>   (call-with-values (lambda ()
>                       (specification->package+output spec)) list))
> 
> (define packages-list
>   '("acpi"
>     "acpica"
>     ...
>     "yasm"
>     "zip"
>   ))
> 
> (packages->manifest
>  (map spec->packages packages-list))
> --8<---------------cut here---------------end--------------->8---
> 
> Now you may want to run cuirass on a dedicated machine and access
> packages built by cuirass on other machines. You just need to run guix
> publish to do that.
> 
> 4) config.scm
> 
> --8<---------------cut here---------------start------------->8---
> (services
>     ...
>     (guix-publish-service #:host "0.0.0.0"))
> --8<---------------cut here---------------end--------------->8---
> 
> 5) other machines config.scm
> 
> --8<---------------cut here---------------start------------->8---
> (services
>  (modify-services %base-services
>                   (guix-service-type
>                    config =>
>                    (guix-configuration
>                     (inherit config)
>                     (substitute-urls '("https://bayfront.guixsd.org";
>                                        "https://mirror.hydra.gnu.org";
>                                        "https://cuirass-machine-name";))))))
> --8<---------------cut here---------------end--------------->8---
> 
> Where "cuirass-machine-name" is the URL of the machine running cuirass.
> 
> Note that the order in substitute list has an importance. Here, the
> substitutes from cuirass-machine-name are only downloaded if they do not
> exist on bayfront and hydra.
> 
> Depending on the upload speed of your build machine you may want to put
> it on top of the list or not ...
> 
> At last, you need to authorize "cuirass-machine-name" key with this
> command :
> 
> --8<---------------cut here---------------start------------->8---
> guix archive --authorize < cuirass-machine-name.pub
> --8<---------------cut here---------------end--------------->8---
> 
> You can refer to
> https://www.gnu.org/software/guix/manual/html_node/Substitutes.html#Substitutes
> for precisions.
> 
> That's it, I hope it will be helful.
> 
> To finish, the downsides of this setup are :
> * You need to keep your manifest up-to-date on your build machine.
> * There are no easy ways to track cuirass status but to do some sql on
> cuirass database.
> 
> Besides that, it's working fine.
> 
> Happy building,
> 
> Mathieu
>