bug#26201: hydra.gnu.org uses ‘guix publish’ for nars and narinfos

[Mark H Weaver]

address@hidden (Ludovic Courtès) writes:

> Hi again!
>
> Until now hydra.gnu.org was using Hydra (the software) to serve not only
> the Web interface but also all the .narinfo and /nar URLs (substitute
> meta-data and substitutes).
>
> Starting from now, hydra.gnu.org directs all .narinfo and corresponding
> nar requests to ‘guix publish’ instead of Hydra.
>
> ‘guix publish’ should be faster and less resource-hungry than Hydra.  It
> uses in-process gzip for nar compression instead of bzip2 (I chose level
> 7, which seems to provide compression ratios close to what bzip2
> provides with its default compression level, while being 3 times
> faster).  Unlike Hydra it never forks so for instance, 404 responses for
> .narinfo URLs should be quicker.  Hopefully, that will improve the
> worst-case (cache miss) throughput.

Excellent!  Any improvement in 404 response time will be very helpful.
I've noticed that spikes of narinfo requests resulting in 404 has been a
major source of overloading on Hydra, because these requests cannot be
cached for very long.  The reason: if we cache those failures for N
minutes, this effectively delays the appearance of new nars by N minutes
(if it was requested before that).  This forces us to choose a small N
for negative cache entries, which means the cache is not much help here.

One question: what will happen in the case of multiple concurrent
requests for the same nar?  Will multiple nar-pack-and-bzip2 processes
be run on-demand?  Recall that the nginx proxy will pass all of those
requests through, and not create the cache entry until it has received a
complete response.  This has caused us severe problems with huge nars
such as texinfo-texmf, to the point that we had to crudely block those
nar requests.  Unfortunately, it is not obvious how to block the
associated narinfo requests due to the lack of job name in the URL, so
this results in failures on the client side that must be manually worked
around.

     Thanks,
       Mark