[Bug-tar] Skipped files with --listed-incremental after rename

[Alex Adriaanse]

Hi there,

This bug may have been previously brought up, but I have not been able to
find it in the archives for this mailing list.  I heard about this bug at
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=159455

To sum things up, when doing an --listed-incremental backup, when a file
gets renamed on a filesystem that does not update the ctime during rename
(such as ReiserFS), tar assumes that the file has already been backed up
under the new name and therefore skips it.  As a result, upon restoration
tar removes the file with the old name and never writes the new file, so the
file essentially gets lost.

This problem doesn't seem to occur on filesystems like ext3, because ext3
updates the file's ctime during rename.  However, when I e-mailed the Linux
kernel mailing list yesterday about this, thinking that ReiserFS ought to
update ctime, I received an e-mail (which I included below) that pointed out
that ReiserFS does not break any standards in this area, because the ctime
doesn't need to be updated when renaming a file.  So it seems that the
problem here is in tar.

I looked through the source of tar 1.13.25 (pinpointing the problem area to
incremen.c lines 300-306), but couldn't figure out an easy to address this
issue without rewriting the way tar saves its snapshot data (having it list
every single file with timestamp and perhaps even checksum, instead of just
directories with timestamps).  Other than doing major modifications to this
part of tar, is there any better way to address this problem?

Thanks,

Alex


-----Original Message-----
From: jw schultz [mailto:address@hidden
Sent: Sunday, October 12, 2003 2:15 AM
To: address@hidden
Subject: Re: ReiserFS patch for updating ctimes of renamed files


On Sun, Oct 12, 2003 at 01:05:19AM -0500, Alex Adriaanse wrote:
> Hi,
>
> I ran into some trouble trying to do incremental backups with GNU tar
> (using --listed-incremental) where renaming a file in between backups
would
> cause the file to disappear upon restoration.  When investigating the
issue
> I discovered that this doesn't happen on ext2, ext3, and tmpfs filesystems
> but only on ReiserFS filesystems.  I also noticed that for example ext3
> updates the affected file's ctime upon rename whereas ReiserFS doesn't, so
> I'm thinking this causes tar to believe that the file existed before the
> first backup was taking under the new name, and as a result it doesn't
back
> it up during the second backup.  So I believe ReiserFS needs to update
> ctimes for renamed files in order for incremental GNU tar backups to work
> reliably.
>
> I made some changes to the reiserfs_rename function that I *think* should
> fix the problem.  However, I don't know much about ReiserFS's internals,
and
> I haven't been able to test them out to see if things work now since I
can't
> afford to deal with potential FS corruption with my current Linux box.
>
> I included a patch below against the 2.4.22 kernel with my changes.  Would
> somebody mind taking a look at this to see if I did things right here (and
> perhaps wouldn't mind testing it out either)?  If it works then I (and I'm
> sure others who've experienced the same problem) would like to see the
> changes applied to the next 2.4.x (and 2.6.x?) release.

Hmm.  I'm conflicted.

rename(2) manpage:
        Any other hard links to the file (as created using
        link(2)) are unaffected.

A change to ctime would affect the other links.

stat(2) manpage:
        The field st_ctime is changed by writing or by
        setting inode information (i.e., owner, group, link
        count, mode, etc.).

I am not aware of any field in the inode structure that must
be changed by an atomic rename.  Per documentation the only
reason rename should update st_ctime is if it does a
link+unlink sequence which would alter st_nlink briefly.

On the other hand it does seem to me there ought to be some
record that something about the inode changed.  st_ctime would
be the only appropriate indicator.

rename() SUSv3:
        Some implementations mark for update the st_ctime
        field of renamed files and some do not. Applications
        which make use of the st_ctime field may behave
        differently with respect to renamed files unless
        they are designed to allow for either behavior.

So reiserfs is on this point definitely standards conformant
already.  A change could at best be seen as an enhancement.




--
________________________________________________________________
        J.W. Schultz            Pegasystems Technologies
        email address:          address@hidden

                Remember Cernan and Schmitt