Re: Caching calls to download-to-store?

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> the CRAN updater downloads every upstream archive three times.  The
>> importer memoizes calls to the download procedure, but one download that
>> is out of reach is the very first.
>>
>> It is triggered by package-update/url-fetch in (guix upstream).  It uses
>> download-to-store, which will unconditionally fetch whatever URL it is
>> given.
>
> Why three times (and not two)?

Not entirely sure, but it's tracked here

   https://issues.guix.gnu.org/48792
   https://issues.guix.gnu.org/44256

I'm already caching some downloads in the updater, but it's not always
working as intended.  (I have an unfinished fix for this locally.)

>> I would very much like to parameterize download-to-store, so that we can
>> optionally cache the results of its invocation.
>
> I’m not sure exactly how to do that, but why not.
>
> Another option would be to register GC roots (or mere symlinks) in
> ~/.cache for things that are downloaded, and then to use
> ‘If-Modified-Since’ plus some default TTL to determine whether to
> re-download things.

That's a good idea.  Currently, we download to arbitrary temporary
directories.  The downloads performed by importers vanish immediately.

-- 
Ricardo