Re: Storing serialised graph along with packages

[Ludovic Courtès]

Hello!

Ricardo Wurmus <address@hidden> skribis:

> it always bothered me that after building a package we lose all of the
> beautiful features that Guix as a Guile library gives us.  We always
> need to keep track of the Guix version at the time of building the
> package and only then can we hope to rebuild the same thing again at
> some point in the future.
>
> What do you think about storing the serialised subset of the package
> graph in a separate output of the package?  Currently, the only place
> where we store anything meta is the database.  Wouldn’t it be great if
> we could “dump an image” of the state of Guile when it has evaluated the
> section of the package graph that is needed to build it?
>
> Then we could just load the serialised state into Guile at a later point
> and inspect the package graph as if we had Guix checked out at the given
> version.  I suppose we could also store this kind of information in the
> database.
>
> I’d really like the graph to stay alive even after Guix has moved on to
> later versions.  It also sounds like a really lispy thing to do.

I sympathize with the goal, and I like the parallel with Lisp.

However I’m skeptical about our ability to do something that is robust
enough.  The package → bag → derivation compilation process is “lossy”
in the sense that at each layer we lose a bit of context from the higher
layers.  Each arrow potentially involves all the code and package
definitions of Guix, as opposed to just a subset of the package
definitions.  We could certainly serialize package objects to sexps, but
that would not capture the implementation of build systems,
‘package-derivation’, or even lower-level primitives.  So this would be
a rough approximation, at best.

The safe way to achieve what you want would be to store the whole Guix
tree (+ GUIX_PACKAGE_PATH), or a pointer to that (a Git commit).

There’s a also the problem of bit-for-bit reproducibility: there’s an
infinite set of source trees that can lead to a given store item.  If we
stored along with, say, Emacs, the Guix source tree/commit that led to
it, then we’d effectively remove that equivalence (whitespace would
become significant, for instance[*].)

Tough issue!

Ludo’.


[*] For gexps we have a similar problem, where we’d like to have source
    code mapping, but that how that mapping is done should not affect
    the .drv itself.