bug#17827: “Too many open files” error while substituter is running

[Ludovic Courtès]

While running the substituter in a disk/VM image built with Guix (on the
metal or on QEMU), I quickly stumble upon EMFILE.  It goes like this:

  1. Run ‘guix build emacs -n’ to warm up /var/guix/substitute-binary;
     this (currently) uses many threads and everything, but doesn’t
     cause any problems.

  2. Run ‘guix build emacs’.  This starts downloading substitutes, via
     ‘guix substitute-binary --substitute’.  It works fine, except that
     running something in parallel like ‘guix package --help’ quickly
     fails with EMFILE (“Too many open files”.)

Looking at /proc/sys/fs/file-nr suggests we’re still way below the
threshold at that point.  Increasing file-max doesn’t make a difference.

Looking at the strace output of the failing ‘guix package --help’ shows
that it does lots of open/close calls, but remains at 10 open files.
Eventually, one of its close(2) calls (!) returns EMFILE, and all the
subsequent open(2) calls fail similarly.

The problem seems to manifest more easily when there’s a lot of RAM,
which made me thing of a Scheme file port leak.

Ludo’.