Making evaluations faster

[Ludovic Courtès]

Hello Guix,

I’ve been looking for ways to make evaluations (computing the derivation
of every package for every supported architecture, like Hydra and
Cuirass do) faster.

I measured evaluations with:

  rm -rf /tmp/cache hydra-jobs.scm
  mkdir /tmp/cache
  XDG_CACHE_HOME=/tmp/cache make hydra-jobs.scm

Currently build-aux/hydra/gnu-system.scm, which performs the evaluation
work, turns on auto-compile such that every Guix module gets compiled.

I measured with 2.2.3 the impact of turning off auto-compilation for
everything but the core modules (meaning that package modules get
interpreted instead) and surprisingly, this is slower than compiling
everything:

  * 2.2.3, auto-compile everything minus (system base compile) & co.

  1362.79user 3.35system 20:59.40elapsed 108%CPU (0avgtext+0avgdata 
1201444maxresident)k
  0inputs+203560outputs (0major+285018minor)pagefaults 0swaps

  * 2.2.3, auto-compile (guix packages) only

  2462.05user 3.69system 39:26.05elapsed 104%CPU (0avgtext+0avgdata 
2121532maxresident)k
  128inputs+36568outputs (0major+242281minor)pagefaults 0swaps

  * 2.2.3, auto-compile ((guix packages) (guix build-system gnu) (guix 
download))

  2364.22user 3.21system 37:44.45elapsed 104%CPU (0avgtext+0avgdata 
2061496maxresident)k
  256inputs+41800outputs (0major+236514minor)pagefaults 0swaps

I guess the extra source properties that are retained when evaluating
account for part the space and time overhead, but I’m not sure this
explains everything.

Andy, what do you think of this?

Thanks,
Ludo’.