Re: [Chicken-users] large amount of data will break open-pipe

[Lui Fungsin]

Hi Felix,

>  Sorry for the overly late reply. I cannot reproduce this. Do you run
>  this code interpreted or compiled? Do I have to press any particular
>  key to provoke the sigpipe?

It is compiled. I tried this on Linux. You just need to enter a big
number (say over 50000) so the chicken program will try to write enough
data to the sink (the pager).

Once the pager started, you press 'q' to quit the pager and the OS
will send a SIGPIPE to the chicken program, which will terminate it.

My question is, whether the WITH-OPEN-PIPE macro, as an abstraction,
should handle SIGPIPE (unlikely). If not, what's the preferred way to
do this handling properly in chicken?

For example, a typical C program will ignore SIGPIPE, and check for
return code for every write/fprintf.

#include <stdio.h>
#include <stdlib.h>

void write_data (FILE * stream)
{
    int i, rc;
    for (i = 0; i < 10000; i++) {
        rc = fprintf(stream,
               "Lorem ipsum dolor sit amet, consectetuer adipiscing
elit. %d\n", i);
        if (rc <= -1) break;
    }
    if (ferror (stream)) {
        fprintf (stderr, "Output to stream failed.\n");
    }
}

int main (void)
{
    FILE *output;

    output = popen ("more", "w");
    signal (SIGPIPE, SIG_IGN);
    write_data (output);
    if (pclose (output) != 0) {
        /* chicken throws exception here */
        fprintf (stderr, "Could not run more or other error.\n");
    }
    printf("exit normally\n");
    return 0;
}


A similar chicken program

(declare (uses extras regex posix utils srfi-13))
(require-extension loop)
(require-extension miscmacros)

(define counter 0)

(define (command-loop)
  (loop for l = (let ((prompt (conc "Enter a number or `exit' :" counter "> ")))
                  (display prompt) (read-line))
        while (and l (not (eof-object? l)) (not (string= l "exit"))) do
        (let ((n (ignore-errors (string->number l))))
          (when n
            (with-output-to-pipe "more"
                 (lambda ()
                   (set! counter 0)
                   (dotimes (i n)
                     (print i " Lorem ipsum dolor sit amet,
consectetuer adipiscing elit.")
                     (set! counter i))))))))

(set-signal-handler! signal/pipe #f)    ;ignore sigpipe

(command-loop)

(print "exit normally")


Try entering a large number, say 50000, then press 'q'.

Even though the pipe is broken, the chicken program will still try to
write to the pipe 50000 times. This is a potential problem because
the body wrapped by with-output-to-pipe can be complex and time
consuming to execute.

I guess this has to do with how r5rs define the write procedure - it
will return an unspecified value, whether it succeeded or not.

After the loop, chicken will throw an exception (cannot close pipe).
If you try to handle this by wrapping a ignore-errors handler around
it, like

(ignore-errors (with-output-to-pipe "more" ......)

sometime it will hang the chicken program - I still need to
investigate why this happens.

I'm thinking whether we could register a sigpipe handler which when
triggered, will update the port sys-slot 8 (closed bool) to #f.

Then we could get behavior like this

#;> (define p (open-output-file "dummy.txt"))
#<output port "dummy.txt">
#;> (close-output-port p)
#;> (display "hi" p)
Error: (display) port already closed: #<output port "dummy.txt">

and the programmer can wrap a handler around the code that does output.

Without this, the only way to handle this now is to wrap a handler
around the whole (with-output-to-pipe ...) code block (and see the
ignore-errors problem described above).

I don't have much to contribute for now. I guess I have to read more
source code to understand the problem better.

Thanks!
-- fungsin