Re: RFC: writable private scratch XDG_CACHE_HOME in build enviroment?

[Andy Wingo]

On Tue 21 Feb 2017 18:50, Mathieu Lirzin <address@hidden> writes:

> Andy Wingo <address@hidden> writes:
>
>>> Since /tmp is writeable in a Guix build environment and in most POSIX
>>> systems (I guess), Would it make sense for Guile itself to fallback to
>>> /tmp for its compilation cache?
>>
>> It's possible.  It's a bit gnarly though:
>>
>>   * you get the usual race conditions between users and /tmp that you
>>     have to mitigate
>
> Nothing different from the race conditions for a single user using
> XDG_CACHE_HOME when compilation is done by multiple threads?

Say you are compiling /foo/bar/baz.scm.  Say your XDG_CACHE_HOME is /tmp
and so it would cache to /tmp/foo/bar/baz.go.  You have to make sure
that /tmp/foo, /tmp/bar, and /tmp/bar.baz.go aren't symlinks created by
another user.

Probably having per-user cache directories fixes this problem for the
leaves, though you would still have it when creating the root
/tmp/mthl-guile-cache directory.

> Maybe we could apply the the sane principles of memoization by computing
> the "value" of a file (by hashing it?) and search it in the different
> ".go" stores (GUILE_LOAD_COMPILED_PATH, XDG_CACHE_HOME, site-ccache,
> ...)?
>
> I don't know if the idea is stupid or not, maybe I miss an obvious
> point.  Anyway this doesn't fit the purpose of fixing your issue in the
> short term.

I would love to do this in Guile but I don't know how :)  Procedural
macros make it tricky.  Consider that there are even some macros that
spawn C compilers!  But there are probably good approximations we can
make (e.g. sha256 of files for imported modules assuming that each
module has a single file, an optional hook to include other files).  I
would like this very much though I don't know yet about the details.

>
>> I'd rather use XDG_CACHE_HOME for this as it's already well specified
>> and actually not Guile-specific
>> (https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).
>
> Setting XDG_CACHE_HOME to the build directory or /tmp in the Guix build
> environnement seems fine to me.

OK I'll do this for the next fibers package and see how it goes.  Thanks
for the feedback.

Andy