Re: [Help-bash] pipe character at end of command ?

[Ulf Andersson A]

My comments interspliced below.

> -----Original Message-----
> From: Eric Blake [mailto:address@hidden
> Sent: den 23 november 2016 17:20
> To: Ulf Andersson A <address@hidden>; address@hidden
> Subject: Re: [Help-bash] pipe character at end of command ?
> 
> On 11/23/2016 09:46 AM, Ulf Andersson A wrote:
> 
> >
> > I am at my wits end. I have searched the bash manual and man page as well
> as numerous wikis and tutorials.
> 
> It's there: look for the section titled Pipelines.
> 

Well, I looked at the section titled Pipelines and it says a lot about 
pipelines but it
says nothing at all about implicit continuation characters. If the mystery 
lines had
had explicit continuation characters, I would have had absolutely no problem
understanding the pipeline whatsoever. But Greg Woodledge explained this
implicit thing in his reply to my question.

> >
> > spunk()
> > {
> >    sed '/Ape/d'    |
> >    sed '/Banana/d' |
> >    sed '/Ladder/d'
> 
> By the way, this form is comparatively expensive (three fork/exec pairs).
> You can achieve the same results with:
> 
> spunk()
> {
>   sed '/Ape/d; /Banana/d; /Ladder/d'
> }

Yes.

> 
> which only forks once.  (In general, MOST cases of piping 'sed' or 'grep' 
> into a
> second 'sed' pass are candidates for evaluation on whether a single 'sed'
> invocation can do all the work itself.)
> 
> > --8><--------------------------------------------
> > I have figured out what the three sed commands do each by themselves,
>> but I have still fo figure out what the pipe characters actually do here.
> 
> Without the pipelines, you are executing three separate command lines,
> each with the same stdin and stdout (the first such command eats all of stdin,
> so the second and third have no input to consume).  With the pipeline, you
> are executing a single command line (the pipe control operator is one of the
> shell operators that works without needing a line continuation backslash);
> within that command line, the stdout of the process on the left of a | is
> hooked to the stdin of the process on the right.  So the sed on Ape sees your
> original stdin, and but only the sed on Ladder writes to your original stdout;
> the other four fds (Ape stdout and Banana stdin, and Banana stdout and
> Ladder stdin) are using the results of the pipe(2) system call to pass data
> between processes without it ever landing on disk.
> 
> > Clearly there is some hand waving with the pipe going on, but as I said
>> above, I have not found any kind of documentation of this behaviour.
> 
> It's documented; it's just that pipes are something that is SO fundamental to
> shell operation that most users already know what they do, so it's hard to
> point out good beginner's documentation to something that so many of us
> already take for granted.

As I said, my problem was not the pipes. I got that. The problem was the bash
behaviour of implicitly adding continuation characters. I didn't get that magic.
It is all much clearer to me now that I have got that magic.

> --
> Eric Blake   eblake redhat com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org