bug#38167: guix pull takes over 8 GiB of memory to finish if there are no substitutes

[Ludovic Courtès]

Hi,

Danny Milosavljevic <address@hidden> skribis:

> On Fri, 27 Dec 2019 19:11:08 +0100
> Ludovic Courtès <address@hidden> wrote:
>
>> Danny Milosavljevic <address@hidden> skribis:
>> 
>> > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>> > dannym   19221 20.8 87.4 9404812 6884184 pts/0 Tl   20:34   2:40 
>> > /gnu/store/sc7z07gim1iq5zvfz1amdwf2irxrzifg-guile-2.2.6/bin/guile 
>> > --no-auto-compile /home/dannym/.config/guix/current/bin/guix pull  
>> 
>> Oh, that’s an RSS of 6 GiB for ‘guix pull’ itself?  Weird, I don’t see
>> how that can happen.
>> 
>> Could it be that ~/.cache/guix/checkouts/THE-THING is too big, which in
>> turn causes libgit2 to consume too much memory somehow?
>
> $ du -hs ~/.cache/guix/checkouts/
> 395M    /home/dannym/.cache/guix/checkouts/

OK, nothing bad.

>> What happens if you attach strace to this process at the moment where
>> it’s consuming a lot of memory?  Is it traversing Git pack files or
>> something like that?
>
> Right now the process is in a paused state and I think when I attach strace
> it will continue.  Should I still do it?  I don't want to destroy our 
> debugging
> opportunity.

I think you should resume it (SIGCONT), attach strace, and grab a few
seconds of log so we have an idea of what it’s doing.  You can also
check /proc/PID/fd and keep a copy of /proc/PID/maps while we’re at it.

Thanks!

Ludo’.