Re: RPC performance

[Andy Wingo]

On Mon 26 Jun 2017 13:54, address@hidden (Ludovic Courtès) writes:

> Andy Wingo <address@hidden> skribis:
>
>> Hi!
>>
>> On Fri 23 Jun 2017 11:24, address@hidden (Ludovic Courtès) writes:
>>
>>> With the current protocol, often we’re just reading a handful of bytes.
>>> Full buffering would mean that Guile would block on an 8K read or so
>>> that will never be fulfilled.
>>
>> That's not how it works :)  The "read" function of a port should only
>> block if no byte can be read.  If 1K bytes are available for an 8K
>> buffer, then the read function should return after filling only 1K
>> bytes; looping to fill at least 8K is some other code's responsibility.
>
> I must be missing something.  With full buffering, when my code does:
>
>   (get-bytevector-n port 8)

Note that get-bytevector-n will block until there are 8 bytes.

> I see read(2) hanging on an 8K read:
>
> #0  0x00007fb0b36baaed in read () at ../sysdeps/unix/syscall-template.S:84
> #1  0x00007fb0b3b91c47 in fport_read (port=<optimized out>, dst=<optimized 
> out>, start=<optimized out>, 
>     count=8192) at fports.c:604
> #2  0x00007fb0b3bbed77 in scm_i_read_bytes (address@hidden, dst=0x195c000, 
> address@hidden, 
>     count=8192) at ports.c:1544
> #3  0x00007fb0b3bc25fe in scm_fill_input (address@hidden, minimum_size=1, 
> address@hidden, 
>     address@hidden, address@hidden) at ports.c:2677

Here this indicates that the buffer is empty, and that it's blocking on
receiving *any* bytes at all.

> (That’s not Guile-specific.)  I agree that read(2) could return less
> than 8K and not block, but it doesn’t have to.

I think this is incorrect.  Read returns when it has any bytes at all.
>From read(2):

   It is not an error if this number is smaller than the number of
   bytes requested; this may happen for example because fewer bytes
   are actually available right now (maybe because we were close to
   end-of-file, or because we are reading from a pipe, or from a
   terminal), or because read() was interrupted by a signal.  See
   also NOTES.

In short the reason read is blocking for you is that there are no bytes
available -- if there were 8 bytes and only 8 bytes, the read(2) would
return directly.

If you have blocking problems related to 8K buffers, it's likely related
to using get-bytevector-n inside CBIP read() functions.

Andy