Latest guile-daemon changes and bewilderment

[Caleb Ristvedt]

Hello guix!

After a pileup of changes and testing, I finally got around to pushing
my changes to the guile-daemon to the branch on savannah. The big
addition is half-working derivation-building (just one at a time, for
now, and only chroot builds for now) in the form of
guix/store/build-derivations.scm. More on the not-working half later.

This includes reference-scanning, which I think pretty much works, but
which I regret to say I spent too much time thinking about. Premature
optimization and all that. It takes a rather different approach to the
way the C++ code does it - it uses a trie to recognize references out of
a list of possibilities (namely, anything thrown in the build
environment's store). The idea is that additional pre-calculation can be
performed on each node and it can become a sort of branching skip table
of the sort used in that one string-matching algorithm whose name I
can't remember. At least, that's the way it works in my head - the parts
to make it fast haven't been implemented yet. I should probably split
the scanner out into a separate module.

The build environment currently is isolated by creating new pid, mount,
ipc, and uts namespaces, remounting everything as MS_PRIVATE, using
pivot-root to make a generated directory the new root, and unmounting
the old root.

So, does it work? Well... sort of. Which brings me to a really bizarre
problem to debug. *deep breath*

The way I've been going about testing this has been to first delete
test-tmp completely, then run (as root, since I'm testing chroot stuff):

./test-env guix build --dry-run hello
./test-env guile
scheme@(guile-user)> (use-modules (ice-9 readline))
scheme@(guile-user)> (activate-readline)
scheme@(guile-user)> ,m (guix store build-derivations)
scheme@(guile-user)> (build-derivation
                      (read-derivation-from-file
                       "path/to/guix/test-tmp/store/...hello-2.10.drv"))

This eventually fails trying to build the bootstrap make. This is the
only error message I get: http://paste.lisp.org/display/351519. So it
seems that some low-level guile thing is failing. I know from
print-statement debugging that the exact place it fails is when the
gnu-build-system calls apply to start the set-paths phase, but have no
indication of why this is. It doesn't make it to the set-paths
procedure, though. Perhaps someone more familiar with guile's internals
knows more?

Builder: http://paste.lisp.org/display/351521.
Environment variables: http://paste.lisp.org/display/351523.

You'll notice that among the environment variables is
GUILE_AUTO_COMPILE=0. This is actually something I added myself because
for some reason the bootstrap guile wasn't honoring the
--no-auto-compile flag, but does honor the environment
variable. Strange.

Stranger still, attempting to run the bootstrap guile interactively from
the build environment (replacing the execl with a system* so the build
environment can be explored from the REPL) causes it to print the
copyright notice and immediately exit with an exit code of 0. So
debugging that way isn't really an option. Notably, running the same
guile from outside of the build environment works fine.

Copy+pasting the builder code into the guile-2.2 REPL in the build
environment after adding the module-import to the load path works just
fine.

Confusing. I know there are several non-essential parts of the build
process that aren't implemented yet, but none of them are used by this
particular derivation as far as I can tell.

- reepca