rdiff-backup-users
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [rdiff-backup-users] Fink rdiff-backup 1.1.0 problems

From: Kevin Horton
Subject: Re: [rdiff-backup-users] Fink rdiff-backup 1.1.0 problems
Date: Tue, 1 Nov 2005 17:59:06 -0500 
On 28-Oct-05, at 21:50 , Ben Escoto wrote:


Kevin Horton <address@hidden>
wrote the following on Fri, 28 Oct 2005 05:23:25 -0400



   File "/sw/lib/python2.4/site-packages/rdiff_backup/metadata.py",
line 247, in get_next_pos

...

IOError: CRC check failed


It seems your most recent mirror_metadata file is corrupt.  You can
check their integrity by by running:

    gzip -t rdiff-backup-data/mirror_metadata.<time here>.gz


You are correct.  The mirror_metadata was corrupt.



So try running 1.1.0 on a clean repository.  If this works with no
issues (and it gets the quoting right) you can try using 1.0.x to make
a backup and then test to make sure 1.1.0 can upgrade it correctly.
I did a fresh backup with the Fink rdiff-backup-1.1.0-3 package. This package uses the following patches: 

# Let Fink install the document files into the proper location instead
# of having rdiff-backup's setup.py do it, which also puts its version
# number in the document directory name.

diff -ru rdiff-backup-1.1.0-orig/setup.py rdiff-backup-1.1.0/setup.py
--- rdiff-backup-1.1.0-orig/setup.py 2004-01-31 20:31:34.000000000 -0800 
+++ rdiff-backup-1.1.0/setup.py 2004-03-08 19:01:56.000000000 -0800
@@ -56,7 +56,4 @@
libraries=libname, extra_link_args=lflags_arg)], 
          scripts = ['rdiff-backup'],
-         data_files = [('share/man/man1', ['rdiff-backup.1']),
- ('share/doc/rdiff-backup-%s' % (version_string,), - ['CHANGELOG', 'COPYING', 'README', 'FAQ.html'])]) 
-
+         data_files = [('share/man/man1', ['rdiff-backup.1'])])

# Patch from
# http://savannah.nongnu.org/cgi-bin/viewcvs/rdiff-backup/rdiff- backup/rdiff_backup/fs_abilities.py.diff?r2=1.21&r1=1.20&diff_format=u 

diff -u -r1.20 -r1.21
--- rdiff-backup-1.1.0/rdiff_backup/fs_abilities.py 2005/10/25 06:43:23 1.20 +++ rdiff-backup-1.1.0/rdiff_backup/fs_abilities.py 2005/10/26 00:52:53 1.21 
@@ -436,8 +436,8 @@
self.update_triple(self.src_fsa.carbonfile, self.dest_fsa.carbonfile, ('carbonfile_active', 'carbonfile_write', 'carbonfile_conn')) if self.src_fsa.carbonfile and not Globals.carbonfile_active: - Log("Source may have carbonfile support, but support defaults to " - "off.\n Use --carbonfile to enable.", 5) + log.Log("Source may have carbonfile support, but support " + "defaults to off.\n Use -- carbonfile to enable.", 5) 
        def set_hardlinks(self):
                if Globals.preserve_hardlinks != 0:
===================================================================
RCS file: /cvsroot/rdiff-backup/cvsroot/rdiff-backup/rdiff-backup/ rdiff_backup/fs_abilities.py,v 
retrieving revision 1.21
retrieving revision 1.22
diff -u -r1.21 -r1.22
--- rdiff-backup-1.1.0/rdiff_backup/fs_abilities.py 2005/10/26 00:52:53 1.21 +++ rdiff-backup-1.1.0/rdiff_backup/fs_abilities.py 2005/10/26 17:42:17 1.22 
@@ -514,7 +514,8 @@
                        return actual_ctq
                log.Log.FatalError("""New quoting requirements!
-The quoting this session appears to need do not match those in
+The quoting chars this session needs (%s) do not match
+the repository settings (%s) listed in
%s
@@ -522,10 +523,7 @@
normal file system onto a windows one that cannot support the same
characters, or if you backup a case-sensitive file system onto a
case-insensitive one that previously only had case-insensitive ones
-backed up onto it.
-
-If you want to risk it, remove the file
-rdiff-backup-data/chars_to_quote.""" % (ctq_rp.path,))
+backed up onto it.""" % (suggested_ctq, actual_ctq, ctq_rp.path))
class RestoreSetGlobals(SetGlobals):

===========================
Items of note:

1. I used a command without any special OS X flags:
sudo rdiff-backup -v7 --print-statistics --exclude-globbing-filelist / rdiff-backup_exclude_list /Users/kwh /Volumes/Ext_BU/Users/kwh/ rdiffbu_test
2. rdiff-backup did the right thing with respect to upper case characters - it detected that both the source and destination were case insensitive, and it did not use any special encoding on the file names. Good. Progress!
3. The info that spews out at the start of the backup indicates that carbonfile support defaults to OFF. I believe the default should be ON (once carbonfile support is working), just in case that info is important to someone. If the fact that this is ON causes an error, and carbonfile support is not important to someone, then they can manually turn it OFF. For now, with carbonfile support not working, I believe the program should present a very visible warning, and pause for user input. The user should be offered the choice to abort, or continue without carbonfile support. The warning should also state that this warning can be avoided in the future by using the --nocarbonfile flag.
4. The backup attempt eventually failed with a “File name too long” error. I’ve got a backup of an old version of iTunes a few directories down from the top, and a Korean language ReadMe has a Unicode file name. I moved this aside for now, but saved it in case you want to do any testing on this case. 

The failure was:
Processing changed file CD for M&D/Updates/iTunes 4.7/Read Before You Install iTunes.app/Contents/Resources/ko.lproj/InfoPlist.strings Regular copying ('CD for M&D', 'Updates', 'iTunes 4.7', 'Read Before You Install iTunes.app', 'Contents', 'Resources', 'ko.lproj', 'InfoPlist.strings') to /Volumes/Ext_BU/Users/kwh/rdiffbu_test/CD for M&D/Updates/iTunes 4.7/Read Before You Install iTunes.app/Contents/ Resources/ko.lproj/rdiff-backup.tmp.6398 Writing file object to /Volumes/Ext_BU/Users/kwh/rdiffbu_test/CD for M&D/Updates/iTunes 4.7/Read Before You Install iTunes.app/Contents/ Resources/ko.lproj/rdiff-backup.tmp.6398 Copying attributes from ('CD for M&D', 'Updates', 'iTunes 4.7', 'Read Before You Install iTunes.app', 'Contents', 'Resources', 'ko.lproj', 'InfoPlist.strings') to /Volumes/Ext_BU/Users/kwh/rdiffbu_test/CD for M&D/Updates/iTunes 4.7/Read Before You Install iTunes.app/Contents/ Resources/ko.lproj/rdiff-backup.tmp.6398 Writing resource fork to ('CD for M&D', 'Updates', 'iTunes 4.7', 'Read Before You Install iTunes.app', 'Contents', 'Resources', 'ko.lproj', 'rdiff-backup.tmp.6398') Setting time of /Volumes/Ext_BU/Users/kwh/rdiffbu_test/CD for M&D/ Updates/iTunes 4.7/Read Before You Install iTunes.app/Contents/ Resources/ko.lproj/rdiff-backup.tmp.6398 to 1096284906 Renaming /Volumes/Ext_BU/Users/kwh/rdiffbu_test/CD for M&D/Updates/ iTunes 4.7/Read Before You Install iTunes.app/Contents/Resources/ ko.lproj/rdiff-backup.tmp.6398 to /Volumes/Ext_BU/Users/kwh/ rdiffbu_test/CD for M&D/Updates/iTunes 4.7/Read Before You Install iTunes.app/Contents/Resources/ko.lproj/InfoPlist.strings Processing changed file CD for M&D/Updates/iTunes 4.7/Read Before You Install iTunes.app/Contents/Resources/ko.lproj/iTunes 설치하기 전에 읽어보기.rtf 
Traceback (most recent call last):
  File "/sw/bin/rdiff-backup", line 23, in ?
    rdiff_backup.Main.Main(sys.argv[1:])
File "/sw/lib/python2.4/site-packages/rdiff_backup/Main.py", line 293, in Main 
    take_action(rps)
File "/sw/lib/python2.4/site-packages/rdiff_backup/Main.py", line 263, in take_action 
    elif action == "backup": Backup(rps[0], rps[1])
File "/sw/lib/python2.4/site-packages/rdiff_backup/Main.py", line 316, in Backup 
    backup.Mirror(rpin, rpout)
File "/sw/lib/python2.4/site-packages/rdiff_backup/backup.py", line 38, in Mirror 
    DestS.patch(dest_rpath, source_diffiter)
File "/sw/lib/python2.4/site-packages/rdiff_backup/backup.py", line 218, in patch 
    ITR(diff.index, diff)
File "/sw/lib/python2.4/site-packages/rdiff_backup/rorpiter.py", line 280, in __call__ 
    if last_branch.can_fast_process(*args):
File "/sw/lib/python2.4/site-packages/rdiff_backup/backup.py", line 470, in can_fast_process 
    rp = self.get_rp_from_root(index)
File "/sw/lib/python2.4/site-packages/rdiff_backup/backup.py", line 465, in get_rp_from_root 
    self.cached_rp = self.basis_root_rp.new_index(index)
File "/sw/lib/python2.4/site-packages/rdiff_backup/rpath.py", line 949, in new_index 
    return self.__class__(self.conn, self.base, index)
File "/sw/lib/python2.4/site-packages/rdiff_backup/ FilenameMapping.py", line 118, in __init__ 
    rpath.RPath.__init__(self, connection, base, quoted_index, data)
File "/sw/lib/python2.4/site-packages/rdiff_backup/rpath.py", line 683, in __init__ 
    else: self.setdata()
File "/sw/lib/python2.4/site-packages/rdiff_backup/rpath.py", line 707, in setdata 
    self.data = self.conn.C.make_file_dict(self.path)
OSError: [Errno 63] File name too long: '/Volumes/Ext_BU/Users/kwh/ rdiffbu_test/CD for M&D/Updates/iTunes 4.7/Read Before You Install iTunes.app/Contents/Resources/ko.lproj/iTunes ; 225;132;137;225;133;165;225;134;175;225;132;142;225;133;181;225;132;146; 225;133;161;225;132;128;225;133;181 ; 225;132;140;225;133;165;225;134;171;225;132;139;225;133;166 ; 225;132;139;225;133;181;225;134;176;225;132;139;225;133;165;225;132;135; 225;133;169;225;132;128;225;133;181.rtf' Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound method GzipFile.__del__ of <gzip open file '/Volumes/Ext_BU/ Users/kwh/rdiffbu_test/rdiff-backup-data/file_statistics. 2005-10-31T19;05849;05841-05;05800.data.gz', mode 'wb' at 0x374d58 0x7a63c8>> ignored Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound method GzipFile.__del__ of <gzip open file '/Volumes/Ext_BU/ Users/kwh/rdiffbu_test/rdiff-backup-data/error_log. 2005-10-31T19;05849;05841-05;05800.data.gz', mode 'wb' at 0x783f50 0x7a9fa8>> ignored Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound method GzipFile.__del__ of <gzip open file '/Volumes/Ext_BU/ Users/kwh/rdiffbu_test/rdiff-backup-data/mirror_metadata. 2005-10-31T19;05849;05841-05;05800.snapshot.gz', mode 'wb' at 0x783f98 0x7a6378>> ignored 
=======================================
I did a fresh BU using rdiff-backup 1.1.0, and another using rdiff- backup 1.0.1. The mirror metadata was corrupt on the fresh backup from v1.0.1 (checked with gzip -t). It was OK on the one from v1.1.0.
I wasn't sure what patches were in my rdiff-backup-1.0.1, so I rebuilt rdiff-backup 1.0.1, using the package in the fink stable tree. No patches, except the standard fink one, that only affects documentation. A fresh backup was made, but mirror metadata was still corrupt.
The fink v1.0.1 package was then modified to use v1.0.2. A fresh backup was made, and the mirror metadata was good. I did an incremental backup using v1.0.2, then updated to fink's rdiff- backup-1.1.0-4 and did an incremental backup using the destination that had previously used v1.0.2. I then did a restore of a large directory with a good selection of file types. The restore was good, and the only item of interest is that the files had lost their association with the programs that created them. The system had reverted to using the file suffix as defining the creator, whereas before many of the files had been identified as belonging to other programs. I believe this is because the backup was made with the carbonfile OFF. This isn't a major issue for me, as the files were still useable. But it is an annoyance. I will do a test with v1.1.0 with carbonfile enabled later to confirm that it is still a problem. 

Conclusions:
1. The Fink rdiff-backup-1.1.0-4 package seems to be working correctly when backing up to and from an HFS+ volume. There is no need to use any special OS X flags. This is a major step forward.
2. The Fink rdiff-backup-1.0.1 package seems to be broken. It should be updated to use v1.0.2, especially as this version has at least one important security fix. 

Kevin Horton
reply via email to
[Prev in Thread] Current Thread [Next in Thread]