Re: Enhancement request for tee - please add the option to not quit on SIGPIPE when someother files are still opened

[Bob Proulx]

Pádraig Brady wrote:
> +If you want to further process the output from process substitutions,
> +and those processes write atomically (i.e. write less than the system's
> +PIPE_BUF size at a time), that's possible with a construct like:

When I read this I read a couple of the sections and it causes me
confusion.  Let me break down my reading of this into parts to
communicate what I am reading.

> +If you want to further process the output from process substitutions,
> +and those processes write atomically

I don't understand the use of "atomically" here.  Use of it causes me
to think of operations that cannot be split into smaller parts and
therefore are completely one way before the action or completely
another way after the action.  I am probably missing something but
that doesn't seem to be the case here.  What action is atomic?  Is
that really what was meant to be communicated?

> +If you want to further process the output from process substitutions,

I have to believe the intention was something regarding "further
process the output" but I fail to understand the intention.  Surely
the output of the process substitutions are just piped along
normally.  I am sure there is a subtle point here that I am missing.

> ... (i.e. write less than the system's
> +PIPE_BUF size at a time), that's possible with a construct like:

When I read this "that is, in other words" clarification indicating
smaller write blocks it makes me wonder what is the atomic operation
in the tee output.  Is there really an atomic operation?  Or is it
simply passing on the data blocks?  That wouldn't really be atomic if
I were to read(0,buf,1) the data, right?

> +@example
> +tardir=your-pkg-M.N
> +tar chof - "$tardir" \
> +  | tee >(md5sum --tag) > >(sha256sum --tag) \
> +  | sort | gpg --clearsign > your-pkg-M.N.tar.sig
> +@end example

I don't see how the example illustrates writes less than PIPE_BUF in
size specifically.

That illustrates my confusion in detail.  And now here let me expand
on other points flowing through my head at this same time.

This does illustrate something that wasn't clear to me when I previous
encountered process substitution.

  $ echo foo | tee >(sleep 2 && cat) > >(sleep 3 && cat) | cat
  $ foo
  foo

The above is asynchronous output.  But adding a pipe means waiting
for the last process to close the write side of the pipe.  This
effectively joins the fork'd processes avoiding the problem.  (Thanks
Pádraig for that construct.)

  $ echo foo | tee >(sleep 2 && cat) > >(sleep 3 && cat) | cat
  foo
  foo
  $

I have always known about the first problem of asynchronous output.  I
had NOT known that adding a pipe would cause them to be wait'd for.
Of course it is waiting for the last writer on the pipe to close the
write end so that the reader can get EOF.  That is quite clever.  It
is very useful.  (Actually I would say that is a required feature to
make process substitutions useful.)  And perhaps subtle enough that it
is lost in the example without a note saying something.  At some point
I could see something like this for synchronization and the reader
will have not know why the pipe to cat and discard is there unless the
author comments it carefully because it is subtle.  Otherwise someone
could naively remove a "| cat >/dev/null" as useless without knowing
it is providing a critical function.

  # Use sleep to simulate long working tasks that emit no output
  # Run some processing in the background and wait for them to finish
  echo | tee >(sleep 2) > >(sleep 30) | cat >/dev/null
  # Background processes are done

And at the same time I hate to try to document it in the tee section
because it is getting pretty far from documenting tee.  It is supposed
to be a document about tee and not an advanced bash programming
manual.  And therefore I am in a quandary about what should be
suggested at that point in the manual.  I am inclined to leave it as
is because I can't think of a way to improve it.

Bob