--- Begin Message ---
|
Subject: |
stdbuf has no effect on some programs |
|
Date: |
Wed, 29 Jun 2011 22:59:13 +0200 |
|
User-agent: |
KMail/1.9.9 |
Hi,
The glibc 'iconv' program buffers its input, and some people don't like this.
I thought that the 'stdbuf' program could remove the buffering, but it does not
work.
How to reproduce:
Create this script and make it executable:
================================== producer ==================================
#!/bin/sh
echo Hello
/bin/sleep 3
echo World
==============================================================================
$ ./producer | /usr/bin/iconv -f ASCII
Hello
World
All the output comes at the end.
$ stdbuf -o 0 ./producer | /usr/bin/iconv -f ASCII
Hello
World
All the output comes at the end.
$ ./producer | stdbuf -i 0 /usr/bin/iconv -f ASCII
Hello
World
All the output comes at the end.
$ stdbuf -o 0 ./producer | stdbuf -i 0 /usr/bin/iconv -f ASCII
Hello
World
All the output comes at the end.
What do I need to do to get the output of the first line immediately?
/usr/bin/iconv is from glibc, but I get the same behaviour from libiconv's
'iconv' program too.
In $ ./producer | /bin/cat
I get the first line immediately, but the coreutils documentation
<http://www.gnu.org/software/coreutils/manual/html_node/stdbuf-invocation.html>
makes me think that 'stdbuf' is meant for those programs that do not work like
'cat'.
Bruno
--
In memoriam José Olaya <http://es.wikipedia.org/wiki/José_Olaya>
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: bug#8961: stdbuf has no effect on some programs |
|
Date: |
Thu, 30 Jun 2011 00:29:22 +0100 |
|
User-agent: |
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 |
On 30/06/11 00:05, Pádraig Brady wrote:
> On 29/06/11 21:59, Bruno Haible wrote:
>> Hi,
>>
>> The glibc 'iconv' program buffers its input, and some people don't like this.
>> I thought that the 'stdbuf' program could remove the buffering, but it does
>> not
>> work.
>
> The following shows I think that iconv is bypassing stdio and buffering
> internally?
>
> (echo; sleep 3; echo) | ltrace iconv -f ASCII
>
> The stdbuf man page notes that:
>
> NOTE: If COMMAND adjusts the buffering of its standard streams (`tee'
> does for e.g.) then that will override corresponding settings changed
> by `stdbuf'. Also some filters (like `dd' and `cat' etc.) don't use
> streams for I/O, and are thus unaffected by `stdbuf' settings.
In fact iconv seems to buffer for ever and so it not scalable,
as demonstrated with this consuming all of memory:
yes | iconv
It would be OK to treat '\n' simply in all
unibyte encodings and utf8 for example,
but that would introduce an inconsistency I suppose.
Though maybe iconv could employ more scalable
buffering at least, for all encodings?
cheers,
Pádraig.
--- End Message ---