Re: Invalid nar signature

[Ludovic Courtès]

David Thompson <address@hidden> skribis:

> Ludovic Courtès <address@hidden> writes:

[...]

>>   2. strace the substituter and/or ‘guix publish’ to see exactly what
>>      happens on the wire.  Is the end-of-file marker string sent?  Is it
>>      received?  etc.
>
> Here's a snippet of the strace output:
>
>   
> http://192.168.1.157/.../iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0
>   94.1% of 127.5 KiB) = 104
>   read(10, "vector tox-max-status-message-le"..., 7728) = 117
>   read(10, "                                "..., 7611) = 1448
>   read(10, "t-last-online (unwrap-tox tox) f"..., 6163) = 1448
>   read(10, "tox tox) nospam))\n\n(define/unwra"..., 4715) = 1448
>   read(10, " friend-number group-number)))\n\n"..., 3267) = 1448
>   read(10, ")))\n    (if (negative? result)\n "..., 1819) = 1819
>   
> http://192.168.1.157/.../iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0
>  100.0% of 127.5 KiB) = 104
>   
> http://192.168.1.157/.../iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0
>  100.0% of 127.5 KiB) = 104
>   brk(0x41875000)                         = 0x41875000
>   mmap(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) 
> = 0x7f3d2aaaf000
>   mremap(0x7f3d2aaaf000, 135168, 266240, MREMAP_MAYMOVE) = 0x7f3d2aa6e000
>   mremap(0x7f3d2aa6e000, 266240, 528384, MREMAP_MAYMOVE) = 0x7f3d2a9ed000
>   mremap(0x7f3d2a9ed000, 528384, 430080, MREMAP_MAYMOVE) = 0x7f3d2a9ed000
>   munmap(0x7f3d2a9ed000, 430080)          = 0
>   
> http://192.168.1.157/.../iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0
>  100.0% of 127.5 KiB) = 104
>   
> open("/gnu/store/72qm7kc9phvsiw6j7xgn1ii0f6s9mx8i-guix-0.8.3b09332/share/locale/en_US.UTF-8/LC_MESSAGES/guix.mo",
>  O_RDONLY) = -1 ENOENT (No such file or directory)
>   
> open("/gnu/store/72qm7kc9phvsiw6j7xgn1ii0f6s9mx8i-guix-0.8.3b09332/share/locale/en_US.utf8/LC_MESSAGES/guix.mo",
>  O_RDONLY) = -1 ENOENT (No such file or directory)
>   
> open("/gnu/store/72qm7kc9phvsiw6j7xgn1ii0f6s9mx8i-guix-0.8.3b09332/share/locale/en_US/LC_MESSAGES/guix.mo",
>  O_RDONLY) = -1 ENOENT (No such file or directory)
>   
> open("/gnu/store/72qm7kc9phvsiw6j7xgn1ii0f6s9mx8i-guix-0.8.3b09332/share/locale/en.UTF-8/LC_MESSAGES/guix.mo",
>  O_RDONLY) = -1 ENOENT (No such file or directory)
>   
> open("/gnu/store/72qm7kc9phvsiw6j7xgn1ii0f6s9mx8i-guix-0.8.3b09332/share/locale/en.utf8/LC_MESSAGES/guix.mo",
>  O_RDONLY) = -1 ENOENT (No such file or directory)
>   
> open("/gnu/store/72qm7kc9phvsiw6j7xgn1ii0f6s9mx8i-guix-0.8.3b09332/share/locale/en/LC_MESSAGES/guix.mo",
>  O_RDONLY) = -1 ENOENT (No such file or directory)
>   write(2, "guix substitute-binary: error: i"..., 62guix substitute-binary: 
> error: invalid nar end-of-file marker
>   ) = 62
>   exit_group(1)                           = ?
>   +++ exited with 1 +++

Normally, when the error happens, the substituter has already created at
least one file in the target directory,
/gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0.

Could you apply the patch below, and then run:

  rm -rf foo
  ./pre-inst-env guix substitute-binary --substitute \
    /gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0 \
    $PWD/foo > stdout
  ls -lRa foo > ls-R

and then send ‘stdout’ and ‘ls-R’?

Could you also check if the files in ‘foo’ look corrupt or anything?

diff --git a/guix/serialization.scm b/guix/serialization.scm
index 64eacf9..d3bdea3 100644
--- a/guix/serialization.scm
+++ b/guix/serialization.scm
@@ -292,19 +292,21 @@ Restore it as FILE."
     (define (read-eof-marker)
       (match (read-string port)
         (")" #t)
-        (x (raise
+        (x
+         (pk 'bad-eof x)
+         (raise
             (condition
              (&message (message "invalid nar end-of-file marker"))
              (&nar-read-error (port port) (file file) (token x)))))))
 
     (match (list (read-string port) (read-string port) (read-string port))
       (("(" "type" "regular")
-       (call-with-output-file file (cut read-contents port <>))
+       (call-with-output-file (pk 'reg file) (cut read-contents port <>))
        (read-eof-marker))
       (("(" "type" "symlink")
        (match (list (read-string port) (read-string port))
          (("target" target)
-          (symlink target file)
+          (symlink target (pk 'symlink file))
           (read-eof-marker))
          (x (raise
              (condition
@@ -312,7 +314,7 @@ Restore it as FILE."
               (&nar-read-error (port port) (file file) (token x)))))))
       (("(" "type" "directory")
        (let ((dir file))
-         (mkdir dir)
+         (mkdir (pk 'dir dir))
          (let loop ((prefix (read-string port)))
            (match prefix
              ("entry"


> Despite that, I tried to compress the nar with bzip2 just for fun, but I
> ran into another problem:
>
>   warning: call to primitive-fork while multiple threads are running;
>            further behavior unspecified.  See "Processes" in the
>            manual, for more information.
>
> I'm running a REPL server in addition to the web server, but I imagine
> the web server also spawns additional threads to handle requests, so
> either way 'filtered-output-port' won't work here.

(web server http) does not use threads so it should be OK.  But we’ll
see that afterwards.  :-)

Thanks,
Ludo’.