Re: GSoC project update 1

[Ludovic Courtès]

Hi reepca!

Caleb Ristvedt <address@hidden> skribis:

> So far, I've replaced the register-path procedure in store.scm, which
> means we now have some support for using sqlite (via guile-sqlite3 and
> some utility macros) to access and modify the state database. We can
> also now perform deduplication on individual store items. That's pretty
> much all that's been produced so far code-wise (that you can see on the
> guile-daemon branch at savannah, anyway), but that's okay, because I've
> also gained a lot of knowledge.

That’s nice work, and a good start!

> Recently I've been trying to understand the details of how
> nix/libstore/build.cc operates, and this has taught me the value of
> trying to keep procedures no longer than one screen's height. It's also
> taught me the value of immediately writing any insights I have down,
> since the details, even of a single procedure I'm reading, are well past
> the point where I can keep them all in my head.
>
> Right now I'm working on implementing the building of derivations. There
> are two main difficulties. The first is that the details of this process
> (which files are included in the chroot by default, permissions,
> ownership, environment variables, etc) are all in C++. I have discovered
> that the picture I had of C++ going into this was, unfortunately, pretty
> accurate. Here's a great example: reading through globals.cc, try to
> figure out what "settings" refers to at each occurrence (I had a long
> rant prepared, but that about sums it up).
>
> The second, less obnoxious difficulty is that of code duplication. A lot
> of the functionality of guix duplicates functionality from the daemon,
> which was one of the motivations for the rewrite in the first place, but
> the duplicated functionality is still built on top of remote procedure
> calls, especially for accessing the database. For example,
> topologically-sorted in store.scm is exactly what I wanted in order to
> implement support for exportReferencesGraph, but it uses an RPC to get
> the list of references of a store item. At the same time, there are
> still many procedures that are what I need (derivations.scm is lovely),
> so it's still worth checking if it's already implemented, but often a
> miss.

As we discussed on IRC, I’m not clear on whether we want to have
RPC-less variants of these procedures, and if we do, how we’d implement
them.

The OO way would be to have a <store> interface, with two classes
implementing it: <local-store> and <remote-store> (that’s what Nix’s
libstore does.)

To do that we could use GOOPS, Guile’s OO system, but I sort of dislike
it (irrationally so, some would say ;-)).  I have a slight preference
for, say, a <store-interface> record type with all the “methods” in it,
but I’m open to suggestions.

> My difficulties with Scheme are pretty minor, and the Guile reference
> has been very useful. It took me awhile to realize that dynamic scoping
> was very much not a thing in Scheme, as opposed to the sort-of-a-thing
> state it's in in Common Lisp (I had a hack set up to shadow a hashing
> function so that I could get the size of the thing being hashed as a
> byproduct, but discovered it didn't work that way and ended up just
> making a wrapper port that counted what went through it). I do get the
> feeling when writing Scheme that set! is an unholy word, which makes for
> a fun challenge, especially where iteration is involved.

Indeed, ‘set!’ is kinda unwelcome, at least in Guix.  Iteration is
usually done via recursive functions as you’ve probably seen, often
using “named let”, as in:

  (let loop ((lst '(1 2 3))
             (result 0))
    (match lst
      (()  ;end of list
       result)
      ((head tail ...)
       (loop tail (+ head result)))))

> I want to finally figure out how monads are supposed to be used,
> though. Perhaps I should just read through the implementation in guix.

Feel free to ask questions as you read through (you’ll probably not need
it in the daemon though).

> I think I have the build environment just about figured out (I've been
> writing a detailed description of the process, which I plan to post once
> it's finished), and can get to implementing derivation-building
> soon. I'll start with assuming that a chroot is used, then extend it to
> work with non-chroot builds later.

Sounds good!  Hopefully ‘call-with-container’ already does most of what
is needed; see how (guix scripts environment) uses it, it’s similar to
what you need.

Thanks for the update!

Ludo’.