bug#10376: [Bug 908354] Re: tail -0f /var/log/kern.log never prints anything (livecd cow overlayfs)

[Sven Breuner]

Jim Meyering wrote on 12/27/2011 10:48 AM:
> Unlike most file system types, overlayFS appears to have no magic number.
> Now we're seeing how using files on such a file system can cause trouble.
> I see no direct way to distinguish this file (for which inotify does not
> work) from any other on a tmpfs file system, for which inotify works
> just fine.
>
> Ugly work-around: tail -f could try using both inotify and polling,
> and, eventually, if polling spots a change for which there was no
> inotify event, it would give up on using inotify for that file.
> As soon as tail sees an inotify event for a file, however, it could
> not stop polling: for most remote FS types, inotify works with changes
> generated locally, but not with those generated remotely.  So polling
> can still be useful when using inotify.


I wouldn't see this as an ugly solution, but rather as a normal hybrid 
approach where tail would just loop and wait with poll(2) on the inotify fd:
- If poll() reports an event, read the file modifications.
- If poll() times out after one second or so, use stat() for the old 
stat-based change detection.
- Either way, poll() again afterwards with the same timeout (i.e. no 
need to disable inotify only because stat() detects a change, which 
inotify didn't detect).

As this would work with local and remote file systems, it could 
completely eliminate the need for different local/remote FS code paths 
in tail (at least from the Linux point of view). It would also bring one 
of the inotify advantages (immediate updates instead of one second 
delay) to the remote file systems, if the writing process and the 
watching process are running on the same machine.

However, the obvious disadvantage here is that tail would need to do 
stat() even on local file systems, where it typically shouldn't be 
necessary at all. (Don't know how important that is.)

Anyways, the actual problem here is of course that overlayFS pretends to 
be another file system, but doesn't behave like applications would 
expect from that other file system. Thus, it's rather a problem with 
overlayFS than a problem with tail - and this might also affect other 
applications, not only tail. So overlayFS should get its own magic number.

Best regards,
Sven