Re: waiting for process substitutions

[Chet Ramey]

On 7/3/24 8:40 PM, Zachary Santer wrote:
> On Wed, Jul 3, 2024 at 11:21 AM Chet Ramey <chet.ramey@case.edu> wrote:
> > Process substitutions are word expansions, with a scope of a single
> > command, and are not expected to survive their read/write file descriptors
> > becoming invalid. You shouldn't need to `wait' for them; they're not
> > true asynchronous processes.
>
> They clearly are. The fact that it results from a word expansion is irrelevant.

They're similar, but they're not jobs. They run in the background, but you
can't use the same set of job control primitives to manipulate them.
Their scope is expected to be the lifetime of the command they're a part
of, not run in the background until they're wanted.


> Consider my original example:
> command-1 | tee >( command-2 ) >( command-3 ) >( command-4 )
>
> Any nontrivial command is going to take more time to run than it took
> to be fed its input.

In some cases, yes.

> The idea that no process in a process
> substitution will outlive its input stream precludes a reading process
> substitution from being useful.

It depends on whether or not it can cope with its input (in this case)
file descriptor being invalidated. In some cases, yes, in some cases, no.

> And nevermind
> exec {fd}< <( command )
> I shouldn't do this?

Sure, of course you can. You've committed to managing the file descriptor
yourself at this point, like any other file descriptor you open with exec.

> To me, a process substitution is just a way to avoid the overhead of
> creating named pipes.

They're not (often) named pipes. What they are is a way to expose an
anonymous pipe in the file system. That may be close to what a named
pipe is, but a /dev/fd filename is not a named pipe (and has some
annoyingly system-specific differences).

> Why should these be different in practice?
>
> (1)
> mkfifo named-pipe
> child process command < named-pipe &
> {
>    foreground shell commands
> } > named-pipe
>
> (2)
> {
>    foreground shell commands
> } > >( child process command )

Because you create a job with one and not the other, explicitly allowing
you to manipulate `child' directly?

> In my actual use cases, I have:
>
> (1)
> A couple different scripts that alternate reading from multiple
> different processes, not entirely unlike
> sort -- <( command-1 ) <( command-2 ) <( command-3 )
> except it's using exec and automatic fds.
>
> Hypothetically, it could work like this:
> {
>    commands
> } {fd[0]}< <( command-1 )  {fd[1]}< <( command-2 ) {fd[2]}< <( command-3 )
> But then again, *I can't get the pids for the processes if I do it this way*.

If you have to get the pids for the individual processes, do it a different
way. That's just not part of what process substitutions provide: they are
word expansions that expand to a filename. If the semantics make it more
convenient for you to use named pipes, then use named pipes.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

OpenPGP_signature.asc
Description: OpenPGP digital signature