GSoC project update 1

[Caleb Ristvedt]

Hello everyone. It's already almost July! Which means I've been working
on replacing the build daemon for about a month now. So it seems like a
good time for an update.

So far, I've replaced the register-path procedure in store.scm, which
means we now have some support for using sqlite (via guile-sqlite3 and
some utility macros) to access and modify the state database. We can
also now perform deduplication on individual store items. That's pretty
much all that's been produced so far code-wise (that you can see on the
guile-daemon branch at savannah, anyway), but that's okay, because I've
also gained a lot of knowledge.

Recently I've been trying to understand the details of how
nix/libstore/build.cc operates, and this has taught me the value of
trying to keep procedures no longer than one screen's height. It's also
taught me the value of immediately writing any insights I have down,
since the details, even of a single procedure I'm reading, are well past
the point where I can keep them all in my head.

Right now I'm working on implementing the building of derivations. There
are two main difficulties. The first is that the details of this process
(which files are included in the chroot by default, permissions,
ownership, environment variables, etc) are all in C++. I have discovered
that the picture I had of C++ going into this was, unfortunately, pretty
accurate. Here's a great example: reading through globals.cc, try to
figure out what "settings" refers to at each occurrence (I had a long
rant prepared, but that about sums it up).

The second, less obnoxious difficulty is that of code duplication. A lot
of the functionality of guix duplicates functionality from the daemon,
which was one of the motivations for the rewrite in the first place, but
the duplicated functionality is still built on top of remote procedure
calls, especially for accessing the database. For example,
topologically-sorted in store.scm is exactly what I wanted in order to
implement support for exportReferencesGraph, but it uses an RPC to get
the list of references of a store item. At the same time, there are
still many procedures that are what I need (derivations.scm is lovely),
so it's still worth checking if it's already implemented, but often a
miss.

My difficulties with Scheme are pretty minor, and the Guile reference
has been very useful. It took me awhile to realize that dynamic scoping
was very much not a thing in Scheme, as opposed to the sort-of-a-thing
state it's in in Common Lisp (I had a hack set up to shadow a hashing
function so that I could get the size of the thing being hashed as a
byproduct, but discovered it didn't work that way and ended up just
making a wrapper port that counted what went through it). I do get the
feeling when writing Scheme that set! is an unholy word, which makes for
a fun challenge, especially where iteration is involved. I want to
finally figure out how monads are supposed to be used, though. Perhaps I
should just read through the implementation in guix.

I think I have the build environment just about figured out (I've been
writing a detailed description of the process, which I plan to post once
it's finished), and can get to implementing derivation-building
soon. I'll start with assuming that a chroot is used, then extend it to
work with non-chroot builds later.

So yeah, that's the state of the project so far.

- reepca