Re: stdbuf feature request - line buffering but for null-terminated data

[Jeffrey Walton]

On Tue, Mar 12, 2024 at 2:59 PM Kaz Kylheku <kaz@kylheku.com> wrote:
>
> On 2024-03-10 01:37, Pádraig Brady wrote:
> > On 09/03/2024 16:30, Zachary Santer wrote:
> >> 'stdbuf --output=L' will line-buffer the command's output stream.
> >> Pretty useful, but that's looking for newlines. Filenames should be
> >> passed between utilities in a null-terminated fashion, because the
> >> null byte is the only byte that can't appear within one.
> >>
> >> If I want to buffer output data on null bytes, the closest I can get
> >> is 'stdbuf --output=0', which doesn't buffer at all. This is pretty
> >> inefficient.
> >>
> >> 0 means unbuffered, and Z is already taken for, I guess, zebibytes.
> >> --output=N, then?
> >>
> >> Would this require a change to libc implementations, or is it possible now?
> >
> > This does seem like useful functionality,
> > but it would require support for libc implementations first.
>
> Here is a lateral thinking kind of idea.
>
> Let's assume that libc is not going to provide a null byte flushing mode any 
> time soon.
>
> What can be done inside utilities so that they can benefit from stdbuf
> flushing control, without having to themselves control buffering?
>
> What if there existed an alternative delimiting mode: a format where
> the character strings are delimited by the two byte sequence \0\n.

One comment here...

Early RFC's from the 1980's also used \r\0. I have not encountered it
in years (decades?), but my "read line" parsing code recognizes it
(just in case).

> This has several advantages:
>
> 1. It now works with line buffering.
>
> 2. Assuming \0 is just an invisible character consumed by terminals with no
>    effect, this format can be dumped to a TTY where it turns into
>    lines, as if the nulls were not there.
>
> 3. Portability: doesn't require a new buffering mode that would only
>    be initially supported in Glibc, and likely never spread beyond
>    a handful of freeware C libraries.

Jeff