Incomplete output from "cvs annotate"

[Kim F. Storm]

In latest CVS emacs on GNU/Linux (redhat9.0), I noticed that the following
sequence: 

        C-h C-n         (open NEWS)
        C-x v g         (annotate it)

produces a very incomplete annotate buffer (approx 2/3 of the output is 
missing).

I then tried the same with 21.2, and got similar, not not quite as bad results.


I have tried to dig into what's going on, but I don't understand...

Here's what I have found out so far:

1) The bug is related to "call-process".

2) I can only make the problem appear with cvs annotate.

3) If I have "cvs -z6" in my ~/.cvsrc the problem is much more severe,
   but it is still present without it.


To narrow down the cause of this error, the following exhibits the error:

In *scratch*, do

        M-x cd (switch to emacs/etc) RET
        (call-process "/usr/bin/cvs" nil t nil "annotate" "NEWS")

This inserts the annotate output directly in *scratch*; the number of
lines inserted should equal the number of lines in NEWS, but it is
more like 4000 than 12400.

If I externally do "cvs annotate NEWS > XYZ", and then call
        (call-process "/bin/cat" nil t nil "XYZ")
in emacs, the entire file is inserted correctly.


I then wrote a small wrapper for cvs annotate:

#include <stdio.h>
#include <errno.h>

main()
{
  FILE *f = popen("/usr/bin/cvs -z6 annotate vc.el", "r");
  int fd = fileno(f);
  char buf[4096*16];
  int xx, tot=0;

  while ((xx = read(fd, buf, sizeof(buf))) > 0)
    {
      fprintf(stderr, "wrote %d[%d]\n", write(1, buf, xx), errno);
    }
  exit(0);
}


When I run this in emacs:

(call-process "mytest" nil t nil)

the output contains some >>wrote 4096[0]<< strings as expected, but
after a while this changes to >> wrote -1[11]<< meaning that the
write would have blocked... 11=EAGAIN ...!?

So what's going on?  As an experiment, I modified the above wrapper to
repeat the write:

#include <stdio.h>
#include <errno.h>

main()
{
  FILE *f = popen("/usr/bin/cvs -z6 annotate vc.el", "r");
  int fd = fileno(f);
  char buf[4096*16];
  int xx, tot=0;

  while ((xx = read(fd, buf, sizeof(buf))) > 0)
    {
      while (xx > 0) {
        int jj = write(1, buf, xx);
        if (jj < 0 && errno == EAGAIN)
          sleep(1);
        else
          xx -= jj;
      }
    }
  
  exit(0);
}


The result is good => the annotate output is now received correctly
and in full by emacs (but of course, very slowly -- that can be tuned
of course).


In a further quest, I wrote a small program which simply wrote
a buffer of 4096 bytes 100 times, and reading the output from
that program (instead of cvs) works with no problems at all
(as do all other examples of using call-process that I have tried).


So the big questions are -- and this is where I'm stuck:

        What is so special with cvs annotate and call-process?

        Why is the pipe opened by call-process -- and which cvs (as
        well as my wrapper around cvs) writes to -- in non-blocking
        state?

        I cannot find the place in emacs_open which sets file modes
        to O_NONBLOCK or O_NDELAY.  Can you?
        

-- 
Kim F. Storm <address@hidden> http://www.cua.dk