Re: Latest guile-daemon changes and bewilderment

[Ludovic Courtès]

Hi reepca!

Caleb Ristvedt <address@hidden> skribis:

> After a pileup of changes and testing, I finally got around to pushing
> my changes to the guile-daemon to the branch on savannah. The big
> addition is half-working derivation-building (just one at a time, for
> now, and only chroot builds for now) in the form of
> guix/store/build-derivations.scm. More on the not-working half later.
>
> This includes reference-scanning, which I think pretty much works,

Woow!

> but which I regret to say I spent too much time thinking
> about. Premature optimization and all that. It takes a rather
> different approach to the way the C++ code does it - it uses a trie to
> recognize references out of a list of possibilities (namely, anything
> thrown in the build environment's store). The idea is that additional
> pre-calculation can be performed on each node and it can become a sort
> of branching skip table of the sort used in that one string-matching
> algorithm whose name I can't remember. At least, that's the way it
> works in my head - the parts to make it fast haven't been implemented
> yet. I should probably split the scanner out into a separate module.

I haven’t put as much thought into the scanning algorithm as you did, so
I’ll defer to you.  Great that you already have a good prototype of
that!

And yes, I agree it’s probably better to have the scanner in its own
module.

> The build environment currently is isolated by creating new pid, mount,
> ipc, and uts namespaces, remounting everything as MS_PRIVATE, using
> pivot-root to make a generated directory the new root, and unmounting
> the old root.

Sounds good.

> So, does it work? Well... sort of. Which brings me to a really bizarre
> problem to debug. *deep breath*
>
> The way I've been going about testing this has been to first delete
> test-tmp completely, then run (as root, since I'm testing chroot stuff):
>
> ./test-env guix build --dry-run hello
> ./test-env guile
> scheme@(guile-user)> (use-modules (ice-9 readline))
> scheme@(guile-user)> (activate-readline)
> scheme@(guile-user)> ,m (guix store build-derivations)
> scheme@(guile-user)> (build-derivation
>                       (read-derivation-from-file
>                        "path/to/guix/test-tmp/store/...hello-2.10.drv"))
>
> This eventually fails trying to build the bootstrap make. This is the
> only error message I get: http://paste.lisp.org/display/351519. So it
> seems that some low-level guile thing is failing. I know from
> print-statement debugging that the exact place it fails is when the
> gnu-build-system calls apply to start the set-paths phase, but have no
> indication of why this is. It doesn't make it to the set-paths
> procedure, though. Perhaps someone more familiar with guile's internals
> knows more?
>
> Builder: http://paste.lisp.org/display/351521.
> Environment variables: http://paste.lisp.org/display/351523.

Is there a line above or below the backtrace mentioning the uncaught
exception?  Could you ‘strace -f’ the daemon process?

Another test would be to manually run the very command that appears in
gnu-make-boot0.drv “by hand”, first outside of a chroot, then within a
chroot with /gnu/store bind-mounted in it (or you could use PRoot for
that.)


BTW, I see the code uses ‘clone’ directly.  It would be safer to use
‘call-with-container’, which already handles bind mounts, non-local
exits, and so on.  Would it be an option?

Regarding scanning, (guix build grafts) contains a special-purpose
reference scanner that Mark carefully optimized; it might be worth
looking at.

> You'll notice that among the environment variables is
> GUILE_AUTO_COMPILE=0. This is actually something I added myself because
> for some reason the bootstrap guile wasn't honoring the
> --no-auto-compile flag, but does honor the environment
> variable. Strange.

Yeah, we shouldn’t add this environment variable to derivations because
that would influence everything that goes in there.

Could it be that ‘argv’ lacked the executable name as argv[0], and thus
the argument list was shifted to the left?

Let’s maybe try to further debug this interactively on IRC.

Thanks for the update!

Ludo’.