Re: Substituting in parallel to save time?

[Ludovic Courtès]

Hi!

Leo Famulari <address@hidden> skribis:

> I noticed that in many cases, it takes longer to set up the substitution
> than to actually download the substitute on my 30 Mbps downlink.

Yeah.

> The effect is that it might take a few minutes to to download a few
> dozen small substitutes, which may only total 5 megabytes of data, even
> though my connection to the substitute server is fast enough that it
> should only take a few seconds.
>
> So, it benefits me to increase max-jobs for substitute downloads.
> However, I rarely want to actually build packages in parallel; one
> package build may fully load my system.
>
> Currently, is it possible to distinguish between substitute downloads
> and builds? Maybe it would be an overall performance win to download
> substitutes in parallel, in tune with parallel-job-count.

No, they cannot be distinguished.

I’d be in favor of a solution where ‘guix substitute’ is kept alive
across substitutions (like what happens with ‘guix substitute --query’),
which would allow it to keep connections alive and thus save the TLS
handshake and a few extra round trips per download.

Thoughts?

Ludo’.