Re: [rdiff-backup-users] Safe to remove old increments while a backup runs?

[Thomas Harold]

On 3/26/2014 7:04 PM, Sam's Lists wrote:
> 
> Anyway, what should I do now? Is there an easy way to tell if my backup
> was corrupted?

You can use the rdiff-backup "--verify" option to test your latest
backup.  To test multiple increments, you have to do a bit of scripting.
 Or feed the dates one at a time into --verify-at arguments.

...

Here is a working fragment from our bash-based backup script.  It is
designed to verify the last 5 increments plus the 'now' increment after
the backup is run.  All of the $RDIFFBACKUP, $SED, $SORT, $GREP
variables are pointers to the various executables.  You will need to
customize this code to match your environment, but it shows how to parse
--list-increments and feed it back into a --verify-at command.

$OFFHOST is the FQDN of our backup server (or can be left blank to
backup to a local file system).  $OFFBASE is the full path to the backup
area, with a trailing '/' on it.  And $DIR is the directory within the
backup area being verified.

--------------------------------------

if [[ ! $VALIDATELASTN =~ ^[0-9]+$ ]]; then VALIDATELASTN=5; fi

RDIFFINCREMENTS=$(
    $RDIFFBACKUP --list-increments ${OFFHOST}::${OFFBASE}${DIR} | \
    $GREP -o 'increments\..*\.dir' | \
    $SED 's/^increments\.//g' | $SED 's/\.dir$//g' | \
    $SORT | $TAIL -n $VALIDATELASTN
    )

RDIFFINCREMENTS+=' now'

echo "$RDIFFBACKUP verify last $VALIDATELASTN increments"
for INCREMENTTIME in ${RDIFFINCREMENTS}; do

    echo "Verify backup at: ${INCREMENTTIME}"

    $RDIFFBACKUP --verify-at "${INCREMENTTIME}" ${OFFHOST}::${OFFBASE}${DIR}

    status=$?
    if [ $status -ne 0 ]; then
        echo "" >> $TMPFILE
        echo "$RDIFFBACKUP verify failed with status: $status"
        echo "$RDIFFBACKUP --verify-at "${INCREMENTTIME}"
${OFFHOST}::${OFFBASE}${DIR} failed with status: $status" >> $TMPFILE
        let ERRCNT+=1
        continue
    fi

done

--------------------------------------

Notes:

Running --verify-at requires a lot of disk activity, especially as you
get deeper and deeper into the past.  This goes much faster if your /tmp
(or wherever you tell rdiff-backup to use as a temporary directory) is
on a different spindle then your backups, or is located on a SSD.

In order to verify a past increment, rdiff-backup has to reconstruct the
file.  The file system where your /tmp directory is located will need to
be large enough to hold the largest file being tested.  A safer rule of
thumb is a 2x or 3x multiplier.  I run into this issue when verifying
backups containing virtual machine images (which are ~30GB).  We were
blowing out the free space on our /tmp which was initially only a 16GB
file system.  We have since moved /tmp to it's own RAID array (RAID-1 on
a pair of Intel SSDs) and made it much larger.