Re: updating list of substitutes

[Mark H Weaver]

Pjotr Prins <address@hidden> writes:

> On Mon, Oct 12, 2015 at 01:15:01AM -0400, Mark H Weaver wrote:
>> The phrase "the substitute list" suggests a single, complete list of all
>> available substitutes, but there is no such list.  Instead, quoting
>> Ludovic above:
>> 
>>   "When building a package FOO, Guix looks for substitutes for FOO
>>    and its prerequisites (those not already available locally.)  It
>>    maintains in /var/guix/substitute/cache a cache of those lookups."
>> 
>> So, if you build package BAR immediately after building FOO, a different
>> set of substitutes is queried, and typically that involves more lookups
>> (unless FOO runtime-depends on BAR).
>> 
>> Does that make sense?
>
> Right. So, why don't we have one list for every build? That would save
> connecting to the one single server every time and be less fragile.

I'm not sure how to make this work well in practice.  One issue is that
the set of available substitutes changes very frequently.  Take a look
at the "Finished at" field of <http://hydra.gnu.org/all> to get an idea.
When Hydra is busy, the set of available substitutes changes every few
minutes, and sometimes several times each minute.  How would we deal
with that?

There are other difficult problems as well, e.g. that the set of
available substitutes includes builds for multiple branches of our git
repo, multiple architectures, and multiple "evaluations" corresponding
to different commits on any given branch of our git repo.

In summary, the full set of available substitutes is typically quite
large and changes frequently, so this approach would entail a lot of
wasted network bandwidth (and load on hydra) to maintain the complete
list of substitutes on every client machine, although only a small
fraction of these would be of interest to any given user.

Does that make sense?

       Mark