[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Possible bug with grep/sed/tail?
From: |
Pádraig Brady |
Subject: |
Re: Possible bug with grep/sed/tail? |
Date: |
Thu, 20 Nov 2008 16:23:45 +0000 |
User-agent: |
Thunderbird 2.0.0.6 (X11/20071008) |
Paolo Bonzini wrote:
>> A) # tail trace.txt | grep "com"
>> - WORKS: produces output
>> B) # tail trace.txt | grep "com" | cat
>> - WORKS: produces output
>> C) # tail -f trace.txt | grep "com"
>> - WORKS: produces output, then waits and reports new lines
>> D) # tail -f trace.txt | grep "com" | cat
>> - FAILS: no output from existing lines, never gets new data
>>
>> To me, it seems completely counterintuitive that A, B, and C would
>> work, but D does not. Each line of input read by tail should be
>> passed to STDOUT, which is then read as STDIN by grep/sed, then passed
>> to STDOUT and read by cat. It should not matter if tail is "done"
>> reading the output or not, as clearly that works fine in case C.
>
> Buffering occurs line by line in cases A and C, in bigger blocks in
> cases B and D. So the data is stuck in grep (or sed's) buffers until
> enough of it is produced. If it is never produced, it is stuck unless
> sed/grep see an end-of-file condition on stdin -- which they do with
> tail, but not with tail -f.
This is a common pain in the neck.
I wrote up a detailed description of the issue:
http://www.pixelbeat.org/programming/stdio_buffering/
Note grep has a --line-buffered option which may help you.
In general if you have a tool in the middle of a pipe line
it will need a way to control the line buffering.
There is a patch at the bottom of the page above which
gives that control to cut.
It's probably useful to add this functionality to all coreutil filters.
I still need to be convinced that glibc is not the correct place for this.
If it was there then any users of stdio could be controlled.
Pádraig.
- Possible bug with grep/sed/tail?, David Corlette, 2008/11/20
- Re: Possible bug with grep/sed/tail?, Paolo Bonzini, 2008/11/20
- Re: Possible bug with grep/sed/tail?,
Pádraig Brady <=
- Re: Possible bug with grep/sed/tail?, Brian Dessent, 2008/11/20
- Re: Possible bug with grep/sed/tail?, Jim Meyering, 2008/11/21
- Re: Possible bug with grep/sed/tail?, Reuben Thomas, 2008/11/21
- Re: Possible bug with grep/sed/tail?, Paolo Bonzini, 2008/11/21
- Re: Possible bug with grep/sed/tail?, Andreas Schwab, 2008/11/21
- Re: Possible bug with grep/sed/tail?, Jim Meyering, 2008/11/21
- Re: Possible bug with grep/sed/tail?, Jim Meyering, 2008/11/22