Re: [Qemu-discuss] vm live storage migration with snapshots

[Jakob Bohm]

On 11/02/2015 20:42, Edward Young wrote:
> Hello Jakob,
>
> Thank you very much for quick and valuable answers to my question!
>
> I have follow up questions about this topic:
>
> 1. Is there any other way we can finish this process without the shared
> storage? I see the last tip you mentioned, we still use a tempsnap in
> shared nfs server. If we consider the long distance VM migration
> scenario, we do not have any shared storage resource between the source
> and destination.

One way would be to make the top snapshot a qcow2 snapshot
created with a vmsave memory snapshot included, force off
the machine (virsh destroy), rsync any leftover changes to
the new locations, then resume the new machine from the
snapshot.  This means downtime.

Another way would be to set up some form of simple file
sharing (NFS, SMB, or similar) for just enough storage to
store the tempsnap small snapshot.  This would typically
be much smaller than the capacity needed to store a full
disk even for a single VM, provided you do the procedure
steps 2-5 (temporary snap, final rsync, migrate, commit)
on one VM at a time.  Assuming of course that nothing
inside the VM decides to do a full disk rewrite (defrag,
zerofree, read-write disk test etc.) in the middle of
the process, as that would explode the temporary
snapshot to full disk size.


> 2. I noticed in the qemu implementation, we have drive_mirror function.
> The drive_mirror will mirror the write requests to both the source and
> the destination. Can we use drive_mirror to transfer the new generated
> data in the migration. In the background, we copy all the snapshots to
> the destination. After the transformation is done, we reconstruct the
> disk images in the destination. I'm not sure whether this is feasible or
> not.

I don't know that option, so cannot advise on its abilities.

> >     *From: *Jakob Bohm <address@hidden
> >     <mailto:address@hidden>>
> >     *Date: *February 11, 2015 11:38:15 AM
> >     *To: address@hidden <mailto:address@hidden>
> >     *Subject: **Re: [Qemu-discuss] vm live storage migration with
> >     snapshots*
> >
> >     On 11/02/2015 18:08, Edward Young wrote:
> > >     Hi all,
> > >
> > >     I'm investigating the ways to improve the live migration
> > >     performance in libvirt. I have a question about the vm live
> > >     storage migration. the platform is libvirt + qemu + kvm
> > >
> > >     If we want to migrate a running vm with its virtual disk images
> > >     to another node.
> > >     we can use 'virsh migrate ....' commands.
> > >
> > >     What if this vm has a number of disk-only external snapshots? In
> > >     the current version, how can live migrate this vm?
> > >
> > >     Is it possible to blockcommit the snapshots to the base image and
> > >     later perform the migration, without shutting down the running vm?
> > >
> > >     Or is it possible to iteratively transfer all the snapshots to
> > >     the destination and later live migrate only the latest new data?
> > >
> > >     Thanks in advance.
> > >
> > >     Best,
> > >     Ed
> >     Just a quick answer, others can answer on clever ways to
> >     transfer disk deltas between systems:
> >
> >     - The usual way is to store both the base images and the
> >      deltas on some kind of shared storage accessible from
> >      both hosts involved in the migration, thus eliminating
> >      the need to transfer any disk contents.
> >
> >     - To the extent that the (not changing at that point in
> >      time) base image and deltas for the non-current
> >      snapshot are stored in separate files, they could
> >      simply be transferred with ordinary tools such as cp, dd
> >      or rsync.
> >
> >     - If you freeze execution of the guest during migration
> >      (not usually a preferred thing when wanting to do live
> >      migration), remaining deltas could be transferred with a
> >      simple rsync.
> >
> >     - If you can find a way to make the snapshot commands
> >      store all further deltas around the time (just
> >      before/after) the migration in a separate delta file on
> >      a shared drive (NFS etc.), you could do that, then rsync
> >      all the (now stable) underlying files, then live migrate
> >      the running VM from using the shared delta and one hosts
> >      underlying files to using the same shared delta with the
> >      other hosts (identical) underlying files, then commit
> >      the snapshot delta from the NFS drive to the new host's
> >      copy of the next lower snapshot or base image.
> >
> >     ASCII art illustration of the final tip above:
> >
> >
> >                 +--------+
> >                 |   VM   |
> >                 +--------+
> >                  /      \
> >                 /4Migrate\
> >         +---------+    +---------+
> >         |  Host1  |    |  Host2  |
> >         +---------+    +---------+
> >           |      \      /      |
> >           | 2Snap \    /5Commit|
> >           |    +----------+    |
> >           |    | TempSnap |    |  On NFS etc.
> >           |    +----------+    |
> >           |      /      \      |
> >           |     / 3rsync \     |
> >         +---------+    +---------+
> >         |Snapshots|    |Snapshots|
> >         +---------+    +---------+
> >              |    1copy     |
> >         +---------+    +---------+
> >         |  Base   |    |  Base   |
> >         +---------+    +---------+
> >
> >        Note that disks are only identical during step 4



Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  http://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded