RE: some concern about the fix of " tail: consistently output all data for truncated files"

[Zizka, Jan (Nokia - CZ/Prague)]

> -----Original Message-----
> From: Reuti [mailto:address@hidden]
> Sent: Wednesday, November 09, 2016 8:43 PM
> Am 09.11.2016 um 09:00 schrieb Zizka, Jan (Nokia - CZ/Prague):
> 
> >> -----Original Message-----
> >> From: Zhang, Bingxuan (Nokia - CN/Hangzhou)
> >> Sent: Wednesday, November 09, 2016 8:51 AM
> >> To: Zizka, Jan (Nokia - CZ/Prague) <address@hidden>; Lian, George
> >> (Nokia - CN/Hangzhou) <address@hidden>; Pádraig Brady
> >> <address@hidden>; address@hidden
> >> Cc: Li, Deqian (Nokia - CN/Hangzhou) <address@hidden>; Bao,
> Xiaohui
> >> (Nokia - CN/Hangzhou) <address@hidden>
> >> Subject: RE: some concern about the fix of " tail: consistently output all
> data
> >> for truncated files"
> >>
> >>> Can you tell any real use case where the changed tail behaviour would
> fail
> >>> and print old content as you describe? I mean some realy use case not
> the
> >>> behaviour caused by GlusterFS bug.
> >>
> >> Not found from real environment, but we can design one program to do
> this:
> >>    A program write a log file, and it want to keep its first 1K bytes
> >> always.
> >>    When the file reach its limit (e.g. 10K bytes), it truncates its content
> >> to 1KB, then start to write content again.
> >>
> >> In this case, with new version, the beginning 1KB data will be printed by
> tail
> >> always when the truncate happen.
> >
> > yes I'm sure one can always find some artificial case, but can you think of
> any real
> > usecase? Because I could not think for any kind of real use case.
> 
> I used `tail -f` in the past to feed the output of the logfile of IBM's Tivoli
> Storage Manager to a remote syslog.
> 
> ITSM can truncate the logfile by keeping only the last e.g. 8 days (no
> rotation), hence the file is getting shorter at one point in time.

ah that is interesting case :) but I think tail could not handle this no matter
how you'd implement it. Here the beggining of file is cut and only last 8 days
of logs remain. Except if tail would really attempt to keep all the data it
piped through and compare :) but that is not I think what tail was ment to do.

Jan

> 
> (Nowadays I implemented this in syslog-ng directly to read the files and
> forward it to a remote syslog-ng server. And yes: syslog-ng has this behavior
> to output the first part of the file again in case it gets truncated. But as 
> I look
> at it only in case of a problem, it wasn't a reason for me to switch back
> again.)
> 
> -- Reuti
> 
> (A sophisticated behavior would be to memorize the already output lines,
> and in case the file gets shorter to scan for a block of at least N matching
> lines to synchronize again - no double output, no missing lines.)
> 
> 
> > Moreover what may happen is that in case of file rotation with old design
> that part
> > of the data will be missing in tail output. And that is real usecase.
> >
> > Jan
> >