Re: Enhancement request for tee - please add the option to not quit on SIGPIPE when someother files are still opened

[Pádraig Brady]

On 20/11/15 04:33, Assaf Gordon wrote:
> Hello Jirka,
> 
> Regarding this:
> 
> On 11/19/2015 08:58 PM, Jirka Hladky wrote:
>>> The general problem I have with >(process substitutions) are that
>>> they are completely asynchronous.  There is no way to tell if they
>>> are done.
>>
>> Yes, I agree with you on this one. However, I don't see the other way
>> how to send the output of one process to multiple sub-processes in
>> shell.
> 
> If I may suggest this slightly verbose shell script (attached) - it should do 
> what you want (sending output to multiple processes)
> while still allowing tight control over each background process, and also 
> collecting their results in an organized fashion
> (ie keeping stdout,stderr,exitcode in a file for each test) - making further 
> diagnosis much easier.
> 
> if there's a need to combine the outputs from all the tests (e.g. to find the 
> smallest p-value from all tests) -  it's just a matter of "cat *.out" once
> all the tests are done.
> 
> Note that this does not solve the "--no-stdout" issue - just the ">()" part. 
> It should also make the shell script portable (except using GNU tee's "-p" 
> parameter).

Note there is no async issue with >() once the output is piped further,
as then the background processes are waited for.
Though yes, using fifos give more fine grained control over processes and exit 
status etc.

> The output should be:
> 
>      tee: standard output: Bad file descriptor
>      == Test 1 exited with code 0 ==
>      == Test 1 STDOUT ==
>      104857600
>      == Test 2 exited with code 0 ==
>      == Test 2 STDOUT ==
>      1
>      == Test 3 exited with code 1 ==
>      == Test 3 STDOUT ==
>      == Test 3 STDERR ==
>      wc: unrecognized option '--foo'
>      Try 'wc --help' for more information.
>      == Test 4 exited with code 0 ==
>      == Test 4 STDOUT ==
>      32768
>      ==
>      Test results stored in /tmp/tmp.esLAoUxeLQ
> 
> 
> Comments and corrections welcomed.

Yes this is a useful pattern.
I noted something similar for use with split(1) at:
http://lists.gnu.org/archive/html/coreutils/2011-05/msg00012.html
with the number of parallel processes potentially determined with nproc(1).

Minor comments on the script. I'd proably `rm -f fifo*` before creating them
to allow clean rerun after Ctrl-C. Also the eval can be simplified to:
  eval TEST_PID=\$TEST${i}_PID

cheers,
Pádraig.