[rdiff-backup-users] rdiff-backup unexpected problem.

[Brice Figureau]

Hi everybody,

I'm running rdiff-backup since more than one year on a bunch of machines
without any problem.

This is between 2 debian sarge machines (librsync 0.9.7 and rdiff-backup
0.13.4).

Since a few days (our Internet connection was accidently down), I can't
backup one of the machine (the other seems to be fine), and all I get is
the following error:

Read from remote host server3: Connection reset by peer
Exception 'Truncated header string (problem probably originated remotely)' 
raised of class 'rdiff_backup.connection.ConnectionReadError':
  File "/usr/lib/python2.3/site-packages/rdiff_backup/robust.py", line 32, in 
check_common_error
    try: return function(*args)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/Rdiff.py", line 89, in 
patch_local
    if outrp: outrp.write_from_fileobj(patchfile)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 947, in 
write_from_fileobj
    copyfileobj(fp, outfp)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 58, in 
copyfileobj
    inbuf = inputfp.read(blocksize)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/librsync.py", line 76, in 
read
    self._add_to_outbuf_once()
  File "/usr/lib/python2.3/site-packages/rdiff_backup/librsync.py", line 85, in 
_add_to_outbuf_once
    if not self.infile_eof: self._add_to_inbuf()
  File "/usr/lib/python2.3/site-packages/rdiff_backup/librsync.py", line 95, in 
_add_to_inbuf
    new_in = self.infile.read(blocksize)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 1155, in 
read
    def read(self, length = -1): return self.file.read(length)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/iterfile.py", line 117, 
in read
    if not self.addtobuffer(): break
  File "/usr/lib/python2.3/site-packages/rdiff_backup/iterfile.py", line 132, 
in addtobuffer
    type, data = self.iwf._get()
  File "/usr/lib/python2.3/site-packages/rdiff_backup/iterfile.py", line 401, 
in _get
    if not self.buf: self.buf += self.file.read()
  File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 513, 
in read
    return self.connection.VirtualFile.readfromid(self.id, length)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 445, 
in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 365, 
in reval
    result = self.get_response(req_num)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 314, 
in get_response
    try: req_num, object = self._get()
  File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 230, 
in _get
    raise ConnectionReadError("Truncated header string (problem "

Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 23, in ?
    rdiff_backup.Main.Main(sys.argv[1:])
  File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 259, in 
Main
    take_action(rps)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 229, in 
take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 278, in 
Backup
    backup.Mirror_and_increment(rpin, rpout, incdir)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/backup.py", line 51, in 
Mirror_and_increment
    DestS.patch_and_increment(dest_rpath, source_diffiter, inc_rpath)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/backup.py", line 226, in 
patch_and_increment
    ITR(diff.index, diff)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/rorpiter.py", line 279, 
in __call__
    last_branch.fast_process(*args)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/backup.py", line 606, in 
fast_process
    if self.patch_to_temp(rp, diff_rorp, tf):
  File "/usr/lib/python2.3/site-packages/rdiff_backup/backup.py", line 498, in 
patch_to_temp
    Rdiff.patch_local, (basis_rp, diff_rorp, new)) == 0: return 0
  File "/usr/lib/python2.3/site-packages/rdiff_backup/robust.py", line 32, in 
check_common_error
    try: return function(*args)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/Rdiff.py", line 89, in 
patch_local
    if outrp: outrp.write_from_fileobj(patchfile)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 947, in 
write_from_fileobj
    copyfileobj(fp, outfp)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 58, in 
copyfileobj
    inbuf = inputfp.read(blocksize)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/librsync.py", line 76, in 
read
    self._add_to_outbuf_once()
  File "/usr/lib/python2.3/site-packages/rdiff_backup/librsync.py", line 85, in 
_add_to_outbuf_once
    if not self.infile_eof: self._add_to_inbuf()
  File "/usr/lib/python2.3/site-packages/rdiff_backup/librsync.py", line 95, in 
_add_to_inbuf
    new_in = self.infile.read(blocksize)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 1155, in 
read
    def read(self, length = -1): return self.file.read(length)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/iterfile.py", line 117, 
in read
    if not self.addtobuffer(): break
  File "/usr/lib/python2.3/site-packages/rdiff_backup/iterfile.py", line 132, 
in addtobuffer
    type, data = self.iwf._get()
  File "/usr/lib/python2.3/site-packages/rdiff_backup/iterfile.py", line 401, 
in _get
    if not self.buf: self.buf += self.file.read()
  File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 513, 
in read
    return self.connection.VirtualFile.readfromid(self.id, length)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 445, 
in __call__
    return apply(self.connection.reval, (self.name,) + args)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 365, 
in reval
    result = self.get_response(req_num)
  File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 314, 
in get_response
    try: req_num, object = self._get()
  File "/usr/lib/python2.3/site-packages/rdiff_backup/connection.py", line 230, 
in _get
    raise ConnectionReadError("Truncated header string (problem "
rdiff_backup.connection.ConnectionReadError: Truncated header string (problem 
probably originated remotely)
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound 
method GzipFile.__del__ of <gzip open file 
'/var/backups/server3/rdiff-backup-data/file_statistics.2005-02-15T12:03:47+01:00.data.gz',
 mode 'wb' at 0x404f74a0 0x405432ec>> ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound 
method GzipFile.__del__ of <gzip open file 
'/var/backups/server3/rdiff-backup-data/error_log.2005-02-15T12:03:47+01:00.data.gz',
 mode 'wb' at 0x40273360 0x405465ac>> ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound 
method GzipFile.__del__ of <gzip open file 
'/var/backups/server3/rdiff-backup-data/mirror_metadata.2005-02-15T12:03:47+01:00.snapshot.gz',
 mode 'wb' at 0x404f74e0 0x4054314c>> ignored

The connection dropped while getting the signature of a file if that can help.
On the remote machine, the rdiff-backup script is still running, long after the 
connection was dropped.

In fact it seems to choke on some files that are quite 'big' (around 50MB).
A tcpdump trace show that the remote server closes the TCP connection (RST), 
thus the Connection reset by peer.
Anyone have an idea of how I can fix that ?
How can I tell the remote site to verbose log everything it does ?

Running on a virgin directory seems to exhibit the same problem.

Thanks for your help,

Brice