Re: [Qemu-discuss] backup procedure validation

[Phil Ehrens]

Hi Raphael,

Sorry, no idea. All I've written is speculation based
on general knowledge. Assuming the nbd device is mature
and behaves similarly to a block device created via, say
'mount -o loop', there shouldn't be any interaction with
the VM beyond normal IO contention.

You should be able to benchmark it's effect easily using
a stupid benchmark like mine, running on the VM:

#!/bin/sh
start=$(date +%s%N)
dd if=/dev/zero of=100M.zero count=200000 2>/dev/null
gzip 100M.zero
sync
stop=$(date +%s%N)
rm -f 100M.zero*
total=$(echo "$stop - $start" | bc)
echo "$total nanoseconds"

Might be interesting to run that on the mounted nbd,
too. Maybe at the same time?

I'm surprised nobody who knows more about this hasn't
chimed in and told me I'm a fool. Any second now...

Phil

Raphael Bauduin wrote:
> Hi Phil,
> 
> Thanks for your answer. Regarding my two other questions, do you know more
> about it?
> - the domain can continue to work normally despite the nbd snapshot ? (Eg
> performance wise, does it have an impact?)
> - when the nbd is disconnected, its snapshot is removed ? (It seems so, but
> I want to be sure I didn't miss anything)
> 
> Raph
> 
> 
> On Wed, Sep 18, 2013 at 6:05 PM, Phil Ehrens <address@hidden>wrote:
> 
> > Hi Raphael,
> >
> > I believe that most people are frightened off
> > by the rsync command line. Same thing that
> > keeps 'screen' from being the most popular
> > application in history.
> >
> > As far as validating the two steps, I believe
> > that using rsync gives you the validation for
> > free. Either the steps are a valid way to create
> > a backup, or rsync will tell you that the source
> > and sink do not match.
> >
> > There's nothing very mysterious about qemu-nbd,
> > it just gives you the necessary block device
> > so that you can point 'mount' at it. Since you
> > are mounting read-only there's not much that
> > can go wrong.
> >
> > Phil
> >
> > Raphael Bauduin wrote:
> > > Hi Phil,
> > >
> > > thanks for your feedback. My intention is indeed not to use the commands
> > > listed as is in a script.
> > > I wanted to validate the procedure of mounting the disk as a nbd snapshot
> > > and rsyncing that.
> > >
> > > I'm asking because this backup procedure seems so simple and easy to set
> > up
> > > compared to qemu-img snapshots that I'm surprised it's not more
> > documented
> > > or discussed.
> > >
> > > Raph
> > >
> > >
> > > On Tue, Sep 17, 2013 at 5:47 PM, Phil Ehrens <address@hidden
> > >wrote:
> > >
> > > > I assume that you intend to script this and run it via cron?
> > > >
> > > > Running a series of commands without testing their return
> > > > values and intelligently parsing stdout/stderr is a formula
> > > > for disaster.
> > > >
> > > > I would run rsync a second time and parse the messages
> > > > returned to be 100% certain of success. rsync is the tool
> > > > for validating rsync.
> > > >
> > > > You should do a simple test to make certain that the
> > > > mount succeeded so that the sync goes to the mount and
> > > > NOT to /mnt/src on the root filesystem.
> > > >
> > > > I would say that >90% of the snapshot/backup scripts that I
> > > > see have fatal flaws. The result is usually either a failure
> > > > to snapshot/backup, or filling up of the wrong filesystem.
> > > >
> > > > Raphael Bauduin wrote:
> > > > > Hi,
> > > > >
> > > > > I found out qem-nbd can mount a snapshot, but this feature is not
> > much
> > > > > documented or discussed (or I didn't find the good resources), so I'd
> > > > like
> > > > > to have some confirmation before I use it for my backups.
> > > > > Here is the sequence of actions I would take:
> > > > >  stop domain
> > > > >  qemu-nbd --connect /dev/nbd0 --read-only --snapshot
> > /path/to/file.qcow2
> > > > >  start domain
> > > > >  mount /dev/nbd0p1 /mnt/src
> > > > >  rsync -a /mnt/src /mnt/dest
> > > > >  umount /mnt/src
> > > > >  qemu-nbd --disconnect /dev/nbd0
> > > > >
> > > > > With these actions, can I be sure that
> > > > > - my backup of the filesystem is correct and coherent  ?
> > > > > - the domain can continue to work normally despite the nbd snapshot ?
> > > > > - when the nbd is disconnected, its snapshot is removed ?
> > > > >
> > > > > Thanks
> > > > >
> > > > > Raphaël
> > > >
> > > > --
> > > > Phil Ehrens <address@hidden> |              mm
> > > > The LIGO Laboratory, MS 100-36     |           /^(  )^\
> > > > California Institute of Technology |           \,(..),/
> > > > 1200 East California Blvd.         |             V~~V
> > > > Pasadena, CA 91125 USA             | My gpg public key:
> > > > Phone:(626)395-8518                | http://www.imbe.net/peligo.asc
> > > >
> > >
> > >
> > >
> > > --
> > > Web database: http://www.myowndb.com
> > > Free Software Developers Meeting: http://www.fosdem.org
> >
> > --
> > Phil Ehrens <address@hidden> |              mm
> > The LIGO Laboratory, MS 100-36     |           /^(  )^\
> > California Institute of Technology |           \,(..),/
> > 1200 East California Blvd.         |             V~~V
> > Pasadena, CA 91125 USA             | My gpg public key:
> > Phone:(626)395-8518                | http://www.imbe.net/peligo.asc
> >
> 
> 
> 
> -- 
> Web database: http://www.myowndb.com
> Free Software Developers Meeting: http://www.fosdem.org

-- 
Phil Ehrens <address@hidden> |              mm
The LIGO Laboratory, MS 100-36     |           /^(  )^\
California Institute of Technology |           \,(..),/
1200 East California Blvd.         |             V~~V
Pasadena, CA 91125 USA             | My gpg public key:
Phone:(626)395-8518                | http://www.imbe.net/peligo.asc