guix-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Making evaluations faster


From: Ludovic Courtès
Subject: Re: Making evaluations faster
Date: Tue, 19 Dec 2017 16:54:09 +0100
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux)

Andy Wingo <address@hidden> skribis:

> On Tue 19 Dec 2017 10:05, address@hidden (Ludovic Courtès) writes:
>
>> I’ve been looking for ways to make evaluations (computing the derivation
>> of every package for every supported architecture, like Hydra and
>> Cuirass do) faster.
>>
>> I measured evaluations with:
>>
>>   rm -rf /tmp/cache hydra-jobs.scm
>>   mkdir /tmp/cache
>>   XDG_CACHE_HOME=/tmp/cache make hydra-jobs.scm
>>
>> Currently build-aux/hydra/gnu-system.scm, which performs the evaluation
>> work, turns on auto-compile such that every Guix module gets compiled.
>>
>> I measured with 2.2.3 the impact of turning off auto-compilation for
>> everything but the core modules (meaning that package modules get
>> interpreted instead) and surprisingly, this is slower than compiling
>> everything:
>>
>>   * 2.2.3, auto-compile everything minus (system base compile) & co.
>>
>>   1362.79user 3.35system 20:59.40elapsed 108%CPU (0avgtext+0avgdata 
>> 1201444maxresident)k
>>   0inputs+203560outputs (0major+285018minor)pagefaults 0swaps
>>
>>   * 2.2.3, auto-compile (guix packages) only
>>
>>   2462.05user 3.69system 39:26.05elapsed 104%CPU (0avgtext+0avgdata 
>> 2121532maxresident)k
>>   128inputs+36568outputs (0major+242281minor)pagefaults 0swaps
>>
>>   * 2.2.3, auto-compile ((guix packages) (guix build-system gnu) (guix 
>> download))
>>
>>   2364.22user 3.21system 37:44.45elapsed 104%CPU (0avgtext+0avgdata 
>> 2061496maxresident)k
>>   256inputs+41800outputs (0major+236514minor)pagefaults 0swaps
>>
>> I guess the extra source properties that are retained when evaluating
>> account for part the space and time overhead, but I’m not sure this
>> explains everything.
>>
>> Andy, what do you think of this?
>
> I have literally no idea :)  This isn't really enough information for me
> to understand things.  I guess my biggest question would be, why are you
> using the auto-compile infrastructure?  Why wouldn't you compile the
> things you want to compile and otherwise turn off auto-compilation?

Evaluations run code from (guix …) as well as everything in (gnu
packages …).

At every evaluation, we have to start afresh: we cannot just reuse .go
files from a previous evaluation, for instance because the ABI might
have changed.  So, to be on the safe side, we re-compile (or
re-evaluate) all these files.

Auto-compilation is the simplest way to do that.  I thought we could
improve performance by compiling only the hot code and interpreting (gnu
packages …), but the measurements above suggest it’s not the case.

I hope that clarifies the situation.  :-)


As an aside, I also feel (though I haven’t tried) that if our packages
were inert sexps, reading them and turning them into <package> records
would be more efficient than it is to compile or interpret our package
modules.  This is understandable in a way (the sexps would be a dumb
language, so no wonder it can be implemented efficiently), but still a
bit puzzling.

Ludo’.



reply via email to

[Prev in Thread] Current Thread [Next in Thread]