Re: substitute derivation: also substitute grafts?

[Csepp]

Maxime Devos <maximedevos@telenet.be> writes:

> [[PGP Signed Part:Undecided]]
>
>
> On 15-09-2022 16:46, Csepp wrote:
>> Ricardo Wurmus <rekado@elephly.net> writes:
>> 
>>> [...]
>>> Did I say *all items*? Well, … grafts are not included, because
>>> graft
>>> derivations are marked as not substitutable.
>>>
>>> Can we change that conditionally? I would really like to avoid
>>> having
>>> to build grafts on B when they have already been built on A.
>> I would love this too, because IO can be incredibly slow on HDDs and
>> large packages.  My netbook would be thankful.
>>
>
> There are some some opportunities for optimizations in the grafting
> code before substituting more -- for example, to avoid seek times, it
> would be possible to rewrite multiple files concurrently (maybe using
> 'par-for-each' to process each file in a directory in parallel).
>
> This is already done in (guix build grafts), except that:
>
>  * 'find-files' itself is not parallelised, even though parallelising it
>     could potentially reduce the time spent seeking (*)
>  * it uses (parallel-job-count), which is (IIUC) 1 by default because
>    --cores=1 by default.  As grafts are more IO intensive than CPU
>    intensive, maybe it would be reasonable to impose a _minimum_ amount
>    of parallelism, I don't, know, 2 or 4 or so?
>
> (I'm assuming the main problem here is seek times.)
>
> Also combinable with the proposal of having substitutes for grafts.
>
> Greetings,
> Maxime.
>
> (*) IIUC, the time for N parallel seeks should, in theory, ≃ 1 seek,
> for small values of N, because of elevator algorithms.
>
> [2. OpenPGP public key --- application/pgp-keys; 
> OpenPGP_0x49E3EE22191725EE.asc]...
>
> [[End of PGP Signed Part]]

Could we store the offsets of references somewhere at build time?