Developing libraries for the GNU system with Guix

[sbaugh]

Hi guix-devel,

When I am hacking on some library Z, I continuously want to test the
effects that my changes to Z have on packages A/B/C which depend on
Z. The same applies, in general, when hacking on any package Z which
other packages A/B/C depend on: While developing, I want to be able to
rapidly mutate Z and see how this affects A/B/C.

I am not sure how to best achieve this. Here are some solutions:

- When you change Z, rebuild A/B/C.
  This is much too slow at present.

- Use grafts to update A/B/C through binary patching.
  This is also too slow, and AFAIK it can't really be sped up.

- Use LD_LIBRARY_PATH to cause A/B/C to search for Z in a mutable place.
  This works for C libraries, but not generically; there are equivalent
  variables for some other languages but it's not a full solution.

- Before starting to hack on Z, build a new version of Z which includes
  a random hash and which A/B/C depend on; then bind-mount a mutable
  directory on top of that. (suggested by mark_weaver on IRC)
  This is the most palatable hack, but still a hack. The inclusion of a
  random hash prevents collision with other packages and the use of
  bind-mounting means we aren't actually mutating the store. This
  unfortunately also requires privileges: it's not usable by
  unprivileged users.

Here are some not currently available possibilities:

- Create some kind of GUIX_LIBRARY_PATH in which packages are looked up
  first before looking at the compiled-in hash.
  This would be an attempt to make a generic equivalent of
  LD_LIBRARY_PATH. This could theoretically be implemented with variant
  symlinks, if they were available: https://lwn.net/Articles/680705/

- Currently every dependency is located at a well known globally unique
  and globally meaningful path; add some kind of "variant package"
  construct which specifies a package which is "passed in" to the
  environment (maybe by bind-mounting it in a filesystem namespace;
  implementation specifics aren't important). To put this in a
  programming language sense, one of these packages being present would
  turn a Guix distribution from a value into a function.

- Massively speed up rebuilding A/B/C by performing incremental
  builds. Not sure how exactly this could work, it's just a thought.

What do you think, guix-devel?

By the way, I think this is a pretty important feature overall (though
it's described in fairly abstract terms in this email). I think this is
one of the last missing features to make the Guix approach a clear and
uncontroversial improvement over existing widespread practices.