February update on data.guix.gnu.org and the Guix Data Service

[Christopher Baines]

Hey,

Another update on the Guix Data Service, I sent out the last update on
the 5th of January [1].

1: https://lists.gnu.org/archive/html/guix-devel/2020-01/msg00073.html

Derivations for system tests [3], as well as channel instances [4]
(which relate to guix pull) are now captured. This is still a work in
progress, I think only the x86_64-linux derivations for the system tests
are captured, and the systems for the channel instances are limited by
what's available in the qemu-binfmt service.

3: 
http://data.guix.gnu.org/revision/3dd311e3a059131ef245417106d4fb659222ef3c/system-tests
4: 
http://data.guix.gnu.org/revision/3dd311e3a059131ef245417106d4fb659222ef3c/channel-instances

The way cross-built derivations are handled has changed. Previously the
system values were used, but I've now tried to move in the direction of
using GNU triplets. This can be seen on the revision pages in the
derivations table [5].

5: http://data.guix.gnu.org/revision/3dd311e3a059131ef245417106d4fb659222ef3c

It's now possible to configure the Guix Data Service to only process
certain branches by listing branches to explicitly include or
exclude. Using this feature, I've configured data.guix.gnu.org to only
look at master as this is what I want this instance of the Guix Data
Service to focus on. Previously, the jobs for core-updates, staging and
other branches could block processing revisions on the master branch, as
often they would take longer to process due to having to build lots of
packages just to build Guix. This issue would have only been amplified
now that data.guix.gnu.org is emulating other architectures and building
Guix for those as well.

Related to this, I've also added some code to enable removing data for a
branch, and removing unreferenced derivations. This is both for cleaning
up the data.guix.gnu.org database now I only want data for master, but
it should also be useful when using the Guix Data Service in an
environment where branches come and go, or be the basis of setting a
retention period for the data.

I forget exactly when, but recently I've been trying to revive the patch
review setup I was working on around a year ago [6]. I've setup an
instance of the Guix Data Service for this [7] (separate to the
data.guix.gnu.org one). I might try and have that instance of the Guix
Data Service process all the branches in the Guix git repository, now
that data.guix.gnu.org doesn't do that.

6: https://lists.gnu.org/archive/html/guix-devel/2019-02/msg00089.html
7: https://guix-patches-data.cbaines.net/

Back to features though, the output from inferior processes used when
loading data for a revision is now captured and stored in the
database. This means you can see more of what's going on, like the
building of libgit2 here for example [8]. Previously you got less
information [9].

8: http://data.guix.gnu.org/job/14657
9: http://data.guix.gnu.org/job/14610

As always, let me know if you have any comments or questions!

Thanks,

Chris

signature.asc
Description: PGP signature