Performance issues with using local-file/interned-file/add-to-store

[Christopher Baines]

Hey,

So I've been playing around with managing some data files with Guix. On
the whole, this is working quite nicely so far, but I'm having some
performance issues with using the local-file gexp.

Using it for large files (~1 to ~4 GB in my case) causes the
guix-daemon to use a large amount of CPU and memory. This isn't a
problem, but as the data files I'm working with don't change, once the
file has been added to the store, it doesn't need to be added again.

The slow performance means that even if nothing needs adding to
the store or building, it can take a while to work that out, as all the
big files that you are using have to be added to the store again anyway.

It would be good for me to have the option to cache the result of
running add-to-store through the local-file gexp, perhaps using the
full filename, or the hash as the cache key?

Is anyone else encountering similar issues, or have ideas about
speeding this up?

Thanks,

Chris

pgpihCgVW2TrJ.pgp
Description: OpenPGP digital signature