Re: [rdiff-backup-users] Forgot A Question

[Andrew K. Bressen]

"Chris Young" <address@hidden> writes:
> I am going to be running rdiff-backup daily from a cron job to backup a
> remote server.
>  
> I want to make sure I am aware of any errors or problems that may have
> occurred each night.
>  
> What files should I look at? Is there a report I can email myself after
> the job runs or are errors just dumped to a main log?

Output goes to stdout and stderr.
I usually run at -v6 which gives python tracebacks if appropriate.

I used to seperate stdout and stderr, but I found I was losing too 
much context, and that the best thing to do is to dump output to
a single file and then send myself the file minus stuff that 
isn't a problem. I changed over to doing it this way about a week
ago, so I'm still testing stuff. 

Currently I have the following scripts in place:

(1)
backups-driver:
#!/bin/bash
# this runs the backup script and greps the log
#
/etc/cronjobs/backup-script >/var/log/old/backups/backup-`date +%Y-%m-%d` 2>&1
#
/etc/cronjobs/backup-report /var/log/old/backups/backup-`date +%Y-%m-%d` | mail 
-s "Backup Report `date +%Y-%m-%d`" address@hidden
#
bzip2 /var/log/old/backups/backup-`date +%Y-%m-%d` 

(2)
backup-script:
 a long set of echo "Starting Machine Foo" followed by rdiff-backup or
 rsync commands. brute force, baby. 

(3)
backup-report:
#!/bin/bash
#
# backup report munger
# akb 05-jun-2003
#
grep -v "^Executing ssh -C" $1 | \
grep -v "^Incrementing mirror file" | \
grep -v "^Touching mirror marker" | \
grep -v "^Cleaning up" | \
grep -v "^Writing hard link data" | \
grep -v "^Regular copying" | \
grep -v "^opening connection using ssh" | \
grep -v "^receiving file list" | \
grep -v "expand file_list" | \
grep -v "^done" | \
grep -v ".* is uptodate" | \
grep -v "^copying unsafe symlink" | \
grep -v ".* is a hard link" | \
grep -v "^backed up" | \
grep -v "^total\: matches\=.*  tag_hits\=.*  false_alarms\=.* data\=.*" | \
grep -v "^wrote .* bytes  read .* bytes  .* bytes/sec" | \
grep -v "^Making directory .*" | \
grep -v "^Processing changed file .*" | \
grep -v "^total size is .*  speedup is .*" 

Since I rsync some machines and rdiff-backup others, a few of the
messages dealt with in the above report script are rsync messages.

Some common messages that still make it thru this are:

1) When backing up a dbm file which is being accessed during the backup
 UpdateError <filename> File changed from regular file before signature

2) When backing up things like syslogs that are changing during backup
 UpdateError <filename> Updated mirror temp file <backupfilename> does not 
match source

3) When a dir has gone away between runs
 Removing directory <dirname>

4) When a file has disappeared during the backup
 Exception '[Errno 2] No such file or directory: 
'/disks/spool/mqueue/dfhA6Hmd5I030432'' raised of class 'exceptions.IOError':
   File "/usr/lib/python2.3/site-packages/rdiff_backup/robust.py", line 32, in 
check_common_error
     try: return function(*args)
   File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 802, in 
open
     else: return open(self.path, mode)

5) When a unix socket file has changed since the last backup
 SpecialFileError run/mysqld/mysqld.sock Socket error: AF_UNIX path too long
 Exception 'Socket error: AF_UNIX path too long' raised of class 
'rdiff_backup.rpath.SkipFileException':
   File "/usr/lib/python2.3/site-packages/rdiff_backup/robust.py", line 32, in 
check_common_error
     try: return function(*args)
   File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 182, in 
copy_with_attribs
     copy(rpin, rpout, compress)
   File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 106, in 
copy
     elif rpin.issock(): rpout.mksock()
   File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 684, in 
mksock
     self.conn.rpath.make_socket_local(self)
   File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 222, in 
make_socket_local
     raise SkipFileException("Socket error: " + str(exc))

I'm still running 0.12.5; I think some of the socket stuff may have 
changed in the 0.13 series. 

The above driver script does seem to occasionally interleave stderr and stdout 
messages that occur at the same time, such as:

  Processing cUpdateError log/kern.log Updated mirror temp file 
/backups/hostname/disk/var/log/rdiff-backup.tmp.52 does not match source
hanged file log
  UpdateError log/mark.log Updated mirror temp file 
/stuff/backups/lanconius.mirror.to/md16/var/log/rdiff-backup.tmp.62 does not 
match source
log

I just noticed this today, and am not sure how to address it; 
folks with more unix shell redirect clue, please? 

I'm supposed to be working on some formatting goals for rdiff-backup
error messages. Will post more about that to seek feedback when I get there.

  --akb