Re: "save" to and "load" from file descriptor

[Olaf Till]

On Wed, Jun 13, 2007 at 07:10:53PM -0400, John W. Eaton wrote:
> On 13-Jun-2007, Olaf Till wrote:
> 
> | work. I cannot enforce binary mode since pipes have only "w" or "r".)
> 
> I think pipes are by definition binary.

But their declared mode is apparantly "w" or "r" at least in some
systems; to my understanding they need not be explicitly "binary",
since that makes no difference on Posix systems.

> 
> | There is shurely no reason to pass variable names, or even to pass
> | more than a single variable in one call. Since both sides of the pipe
> | are usually programmed by the same person, if he/she likes, multiple
> | variables and variable names can be put into a cell array or a
> | structure before.
> | 
> | So I would accept only a single variable to pass, and pass its value
> | itself, and in loading return the value, not install the named
> | variable into memory. The name used with save_binary_data() would be
> | an arbitrary one than.
> 
> If the values are not named in some way, then how does the receiving
> end know what values it is receiving?  It seems that somehow that
> information would need to be transferred.  It doesn't seem like a very
> flexible solution if the data that you are transferring has a fixed
> order that must be agreed by the programs before they start.  So it
> seems that you need more than just the abilitly to hook up the save
> and load functions to a pipe, and that you need to define a protocol
> or some sort to define what data is being sent, in addition to sending
> the data.
> 

I saw it so: The receiving end must know what variables it expects
anyway. If this flexibility is needed, names or orders can be stored
in a cell array, which is a field of a structure, the other field
would be a cell array with the data variables.

But additionally the name could be tranferred and the variable
installed.

> | ...
> | psend (pid{2}, var1);
> | psend (pid{2}, var2);
> | ...
> | pclose (pid{2});
> | 
> | would be allowed than, and the corresponding prcv (pid{1}) would have
> | to be called several times in this case.
> 
> You will still need a way to send the header information, or you will
> have to restrict yourself to a particular format, byte order, etc.

A restriction to the Octave binary format seemed reasonable to me. Did
not consider float format, byte order etc. So really a header must be
sent and loaded. Luckily the respective functions seem to be exported,
too.

This is a reason to allow only one variable per call. Otherwise, since
the sender can make more than one "psend" call before closing, the
receiver could not easily tell if it has to expect a header or a
variable after having read the first variable.


> 
> | Since e.g.
> | 
> | psend (pid{2}, var1);
> | fprintf (pid{2}, "something", ...);
> | ...
> | pclose (pid{2})
> | 
> | should also be possible, and the other side has to parse the stream
> | "manually" in this case, there should also be a pendant to "prcv()"
> | that reads from a string.
> 
> I think that would just be an error.  If you allow arbitrary data
> between blocks of data that are supposed to contain Octave values,
> then I think it would probably not be reliable.
> 

The receiving end has to parse the stream and must detect what is
arbitrary data (by some convention of its format). The rest in between
are octave variables in binary format, they can be stored in
strings. These strings could be fed into read_binary_data() (after
reading the header). At least so I thought.

But there would be another way, without a string parser: if there was
an interpreted tool to peek some bytes from the streem (pipe), these
bites could be examined if they mark data which is no octave variable
(the size of these data must be given, too, by a userprogram-internal
convention). If yes, these data can be fully read. If not, the next
Octave variable can be tried to read from the pipe.

Olaf