Re: [rdiff-backup-users] Warning, metadata file has entry for ...

[rdiff]

Chris,

Before you continue, backup the entire backup tree (data and increments). 
Fiddling could make you loose your increments forever if you don't.  Once 
you have a backup, try a --check-destination-dir with the latest version 
(and, what the heck, you have a backup, try devel too).  If that still 
doesn't work, read on...

Someone correct me if i'm wrong here -- If the drive filled up, then what 
probably happend is ...

Most (half, some, whatever) of the files were transferred, updated and had 
increments put in place.  The rest of the files are from the previous 
sucessful backup (the one before it failed) which have no increments from 
the backup which filled the drive (most recent).

The files which do have increments from the backup which filled the drive 
need to have the increments applied in reverse to put the files back as 
they were so that their previous rdiff increments will be valid.  I have a 
feeling that is what regression does and if it won't even regress, then 
I'm not sure.

There's probably a file in your increment tree which is causing problems 
and making the regression bomb.   Let's reduce the playing field by 
finding rdiffs which are bad gzip files:

find /backupdir/rdiff-backup-data | grep gz$ | while read file; do
        if ! zcat "$file" > /dev/null ; then
                echo "$file is bad"
        fi
done

You might also find files of zero size:

find /backupdir/rdiff-backup-data/ |  egrep -v '(missing|dir)$' | while read 
file ; do
        if ! [ -s "$file" ]; then
                echo "$file is zero bytes"
        fi
done

This files are /probably(maybe)/ problem files.  Now what to do with them, 
I'm really not sure.  Backup your whole tree before trying this, but you 
might delete damaged .gz files and zero byte files which you know should 
be >0 bytes and then running --check-destination-dir again.  Bottom line 
is that you need to find what is causing the regression to fail clean it 
up.

Finding where the backup died might be helpful as well.  This will show 
you the files in file file modification (mtime) order.  Perl's stat()[9] 
is mtime:

find backupdir/ | \
        perl -lne '@a = stat($_); print "$a[9]: $_"' | sort -n

The last files were modified most recently and so they might have 
something to do with failing the regression.  This assumes your mtimes are 
intact and that you have a good clock on your system ;)

Once again, I'm not really sure what to do once you find the offending 
files... this is hopefully just some direction for you to follow.  Someone 
else please chime in if you have ideas!

-Eric



On Fri, 12 May 2006, Chris Wilson wrote:

> Hi all,
>
> I'm getting very worried by this. I have no backups for the last two months, 
> because rdiff-backup keeps crashing with this error every day. My only option 
> right now is to blow away a year's history with the rdiff-backup-data 
> directory. I don't know Python so I can't diagnose it myself. Can anyone 
> help, or tell me what more information I need to provide to fix this?
>
> Cheers, Chris.
>
> On Sat, 1 Apr 2006, Chris Wilson wrote:
>
> > Hi all,
> >
> > > On the machine where the disk filled up, I'm now getting the following
> > > errors every time I run a particular backup:
> [...]
> > The problem seems to be that the current backup will not complete 
> > successfully, so I get the same warnings next time.
> >
> > The backup aborts with this error:
> >
> > Warning, metadata file has entry for 
> > tmp/home/chris/lex/WEB-INF/classes/com/qwirx/lex/WrongTypeError.class,
> > but there are no associated files.
> > Traceback (most recent call last):
> >  File "/usr/bin/rdiff-backup", line 23, in ?
> >    rdiff_backup.Main.Main(sys.argv[1:])
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line 285, in 
> > Main
> >    take_action(rps)
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line 257, in 
> > take_action
> >    elif action == "check-destination-dir": CheckDest(rps[0])
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line 861, in 
> > CheckDest
> >    dest_rp.conn.regress.Regress(dest_rp)
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line 70, 
> > in Regress
> >    for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line 162, 
> > in iterate_meta_rfs
> >    for raw_rf, metadata_rorp in collated:
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/rorpiter.py", line 92, 
> > in Collate2Iters
> >    try: relem1 = riter1.next()
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line 140, 
> > in helper
> >    for sub_sub_rf in helper(sub_rf):
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line 140, 
> > in helper
> >    for sub_sub_rf in helper(sub_rf):
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line 140, 
> > in helper
> >    for sub_sub_rf in helper(sub_rf):
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line 140, 
> > in helper
> >    for sub_sub_rf in helper(sub_rf):
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line 140, 
> > in helper
> >    for sub_sub_rf in helper(sub_rf):
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line 140, 
> > in helper
> >    for sub_sub_rf in helper(sub_rf):
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line 140, 
> > in helper
> >    for sub_sub_rf in helper(sub_rf):
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line 139, 
> > in helper
> >    for sub_rf in rf.yield_sub_rfs():
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/restore.py", line 546, 
> > in yield_sub_rfs
> >    yield self.__class__(mirror_rp, inc_rp, inc_list)
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line 182, 
> > in __init__
> >    self.set_regress_inc()
> >  File "/usr/lib/python2.4/site-packages/rdiff_backup/regress.py", line 197, 
> > in set_regress_inc
> >    assert len(newer_incs) <= 1, "Too many recent increments"
> > AssertionError: Too many recent increments
> >
> > The same thing happens when I run --check-destination-dir on the target
> > directory.
> >
> > I don't know Python and I have no idea what's causing this assertion
> > failure (although I imagine it's due to corruption in the metadata). Can
> > anyone help?
> >
> > I'd really rather not wipe out the rdiff-backup-data directory and lose
> > all my history, since that's why I use rdiff-backup in the first place.
> >
> > Cheers, Chris.

--
Eric Wheeler
Vice President
National Security Concepts, Inc.
PO Box 3567
Tualatin, OR 97062

http://www.nsci.us/
Voice: (503) 293-7656
Fax:   (503) 885-0770