bug#20402: HTTP and FTP proxies for guix package / guix-daemon

[Ludovic Courtès]

"Claes Wallin (韋嘉誠)" <address@hidden> skribis:

>>> But guix-daemon still doesn't seem to be honoring http_proxy.
>>
>> As a first test, could you check whether ‘guix download’ honors it?
>
> It does.
>
>> A second test would be:
>>
>>   1. Define ‘http_proxy’ in your user shell.
>>   2. From the same shell, run ‘guix build -S gcc --no-substitutes’, say.
>>
>> Does that honor ‘http_proxy’?
>
> http_proxy is set (and exported) in user shell and in the daemon process.

Oops, my instructions were incorrect: ‘http_proxy’ needs to be defined
in the daemon process.

To test, I first run the daemon like this:

  sudo sh -c 'export http_proxy=foobar ; ./pre-inst-env guix-daemon 
--build-users-group=guixbuild'

and then I run:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build -S gcc --no-substitutes
accepted connection from pid 14399, user ludo
La jenaj derivoj estos konstruataj:
   /gnu/store/s32s498y88sidfnrppsn33nfmhlacpzx-gcc-5.1.0.tar.xz.drv
   /gnu/store/z1n2c3hgl7w2cnz9d5szyjwwy8lp0dav-gcc-5.1.0.tar.bz2.drv
@ build-started 
/gnu/store/z1n2c3hgl7w2cnz9d5szyjwwy8lp0dav-gcc-5.1.0.tar.bz2.drv - 
x86_64-linux 
/var/log/guix/drvs/z1//n2c3hgl7w2cnz9d5szyjwwy8lp0dav-gcc-5.1.0.tar.bz2.drv.bz2
starting download of 
`/gnu/store/6a3d39111w0d5n7rzb5p7aliilw0hx1l-gcc-5.1.0.tar.bz2' from 
`http://ftpmirror.gnu.org/gcc/gcc-5.1.0/gcc-5.1.0.tar.bz2'...
ERROR: In procedure struct_vtable: Wrong type argument in position 1 (expecting 
struct): #f
starting download of 
`/gnu/store/6a3d39111w0d5n7rzb5p7aliilw0hx1l-gcc-5.1.0.tar.bz2' from 
`ftp://ftp.cs.tu-berlin.de/pub/gnu/gcc/gcc-5.1.0/gcc-5.1.0.tar.bz2'...
ftp://ftp.cs.tu-berlin.de/.../gcc-5.1.0.tar.bz2   3.9% of 92728.9 KiB (610. 
KiB/s)  C-c C-cSIGPOLL
--8<---------------cut here---------------end--------------->8---

Here the error message comes from the fact that $http_proxy is a
malformed URL, showing that it *is* honored.  Since the http URL fails,
the download logic falls back to the next URL, which happens to be an
ftp URL, which is unaffected by ‘http_proxy’.

So it’s working as expected here.

>> And a third test is for substitutes: Are they also downloaded via the
>> proxy?  Here you need to:
>>
>>   1. Define ‘http_proxy’ in a root shell.
>>   2. Run ‘guix-daemon’ from this shell.
>>   3. From a user shell, run ‘guix build gcc’, say, which should normally
>>      download a substitute from hydra.gnu.org rather than trying to build
>>      things.
>>
>> Does the substitute download honor ‘http_proxy’?

This test also works as expected here:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build -S inkscape
accepted connection from pid 15118, user ludo
La jena derivo estos konstruata:
   /gnu/store/fa6ss1ljwb97ns75332nshskj2bv0dh4-inkscape-0.91.tar.gz.drv
La jenaj derivoj estos elŝutataj:
   /gnu/store/acmzy3nkms38g6vrphh07bczf5s5mbjd-gnutls-3.4.1
   /gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1
@ substituter-started /gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1 
/home/ludo/src/guix/nix/scripts/substitute
found valid signature for 
'/gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1', from 
'http://hydra.gnu.org/nar/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1'
downloading `/gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1' (1.7 MiB 
installed)...

[...]

In guix/scripts/substitute.scm:
 182: 2 [#<procedure 1523210 at guix/scripts/substitute.scm:165:9 ()>]
In guix/build/download.scm:
 212: 1 [open-socket-for-uri #]
In web/client.scm:
  73: 0 [open-socket-for-uri #]

web/client.scm:73:16: In procedure open-socket-for-uri:
web/client.scm:73:16: In procedure struct_vtable: Wrong type argument in 
position 1 (expecting struct): #f
@ substituter-failed /gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1 
256 fetching path `/gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1' 
failed with exit code 1
guix build: error: build failed: some substitutes for the outputs of derivation 
`/gnu/store/2v6lls3f8dj1rkpram4jnl8ckqs77r6m-gnutls-3.4.1.drv' failed (usually 
happens due to networking issues); try `--fallback' to build derivation from 
source
--8<---------------cut here---------------end--------------->8---

The backtrace again stems from the bogus $http_proxy that I chose (not
nice, but shows that $http_proxy is honored.)

Please let me know if you have evidence that something works differently
for you.

> No. Whenever the daemon is doing the downloading, http_proxy is
> ignored and I get "Temporary failure in name resolution" because the
> VM doesn't have an internet DNS.

[...]

> `/gnu/store/2zkyyw4bq0wzsxdysncrf9lmwl44w5wh-binutils-2.25.tar.bz2'
> from `http://ftpmirror.gnu.org/binutils/binutils-2.25.tar.bz2'...
> ERROR: In procedure getaddrinfo: Temporary failure in name resolution

That’s another, unrelated issue, probably in your network or on your
machine.

Thanks,
Ludo’.