Re: [Qemu-devel] [PATCH v5 00/12] Dirty bitmaps migration

[Vladimir Sementsov-Ogievskiy]

On 03.06.2015 01:17, John Snow wrote:
> On 05/28/2015 04:56 PM, Denis V. Lunev wrote:
> > On 28/05/15 23:09, John Snow wrote:
> > > On 05/26/2015 10:51 AM, Denis V. Lunev wrote:
> > > > On 26/05/15 17:48, Denis V. Lunev wrote:
> > > > > On 21/05/15 19:44, John Snow wrote:
> > > > > > On 05/21/2015 09:57 AM, Denis V. Lunev wrote:
> > > > > > > On 21/05/15 16:51, Vladimir Sementsov-Ogievskiy wrote:
> > > > > > > > Hi all.
> > > > > > > >
> > > > > > > > Hmm. There is an interesting suggestion from Denis Lunev (in CC)
> > > > > > > > about
> > > > > > > > how to drop meta bitmaps and make things easer.
> > > > > > > >
> > > > > > > > method:
> > > > > > > >
> > > > > > > > > start migration
> > > > > > > > disk and memory are migrated, but not dirty bitmaps.
> > > > > > > > > stop vm
> > > > > > > > create all necessary bitmaps in destination vm (empty, but with same
> > > > > > > > names and granularities and enabled flag)
> > > > > > > > > start destination vm
> > > > > > > > empty bitmaps are tracking now
> > > > > > > > > start migrating dirty bitmaps. merge them to corresponding bitmaps
> > > > > > > > in destination
> > > > > > > > while bitmaps are migrating, they should be in some kind of
> > > > > > > > 'inconsistent' state.
> > > > > > > > so, we can't start backup or other migration while bitmaps are
> > > > > > > > migrating, but vm is already _running_ on destination.
> > > > > > > >
> > > > > > > > what do you think about it?
> > > > > > > the description is a bit incorrect
> > > > > > >
> > > > > > > - start migration process, perform memory and disk migration
> > > > > > >       as usual. VM is still executed at source
> > > > > > > - start VM on target. VM on source should be on pause as usual,
> > > > > > >       do not finish migration process. Running VM on target "writes"
> > > > > > >       normally setting dirty bits as usual
> > > > > > > - copy active dirty bitmaps from source to target. This is safe
> > > > > > >       as VM on source is not running
> > > > > > > - "OR" copied bitmaps with ones running on target
> > > > > > > - finish migration process (stop source VM).
> > > > > > >
> > > > > > > Downtime will not be increased due to dirty bitmaps with this
> > > > > > > approach, migration process is very simple - plain data copy.
> > > > > > >
> > > > > > > Regards,
> > > > > > >        Den
> > > > > > I was actually just discussing the live migration approach a little
> > > > > > bit
> > > > > > ago with Stefan, trying to decide on the "right" packet format (The
> > > > > > only
> > > > > > two patches I haven't ACKed yet are ones in which we need to choose a
> > > > > > send size) and we decided that 1KiB chunk sends would be
> > > > > > appropriate for
> > > > > > live migration.
> > > > > >
> > > > > > I think I'm okay with that method, but obviously this approach
> > > > > > outlined
> > > > > > here would also work very well and would avoid meta bitmaps, chunk
> > > > > > sizes, migration tuning, convergence questions, etc etc etc.
> > > > > >
> > > > > > You'd need to add a new status to the bitmap on the target (maybe
> > > > > > "INCOMPLETE" or "MIGRATING") that prevents it from being used for a
> > > > > > backup operation without preventing it from recording new writes.
> > > > > >
> > > > > > My only concern is how easy it will be to work this into the migration
> > > > > > workflow.
> > > > > >
> > > > > > It would require some sort of "post-migration" ternary phase, I
> > > > > > suppose,
> > > > > > for devices/data that can be transferred after the VM starts -- and I
> > > > > > suspect we'll be the only use of that phase for now.
> > > > > >
> > > > > > David, what are your thoughts, here? Would you prefer Vladimir and I
> > > > > > push forward on the live migration approach, or add a new post-hoc
> > > > > > phase? This approach might be simpler on the block layer, but I
> > > > > > would be
> > > > > > rather upset if he scrapped his entire series for the second time for
> > > > > > another approach that also didn't get accepted.
> > > > > >
> > > > > > --js
> > > > > hmmm.... It looks like we should proceed with this to fit 2.4 dates.
> > > > > There is not much interest at the moment. I think that we could
> > > > > implement this later in 2.5 etc...
> > > > >
> > > > > Regards,
> > > > >       Den
> > > > oops. I have written something strange. Anyway, I think that for
> > > > now we should proceed with this patchset to fit QEMU 2.4 dates.
> > > > The implementation with additional stage (my proposal) could be
> > > > added later, f.e. in 2.5 as I do not see much interest from migration
> > > > gurus.
> > > >
> > > > In this case the review will take a ... lot of time.
> > > >
> > > > Regards,
> > > >       Den
> > > That sounds good to me. I think this solution is workable for 2.4, and
> > > we can begin working on a post-migration phase for the future to help
> > > simplify our cases a lot.
> > >
> > > I have been out sick much of this week, so apologies in my lack of
> > > fervor getting this series upstream recently.
> > >
> > > --js
> > no prob :)
> Had a chat with Stefan about this approach and apparently that's what
> the postcopy migration patches on-list are all about.
>
> Stefan brought up the point of post-hoc reliability: It's possible to
> transfer control to the new VM and then lose your link, making migration
> completion impossible. Adding a post-copy phase to our existing live
> migration is a non-starter, because it introduces unfairly this
> unreliability to the existing system.
>
> However, we can make this idea work for migrations started via the
> post-copy mechanism, because the entire migration already carries that
> known risk of completion failure.
>
> It seems like the likely outcome though is that migrations will be able
> to be completed with either mechanism in the future: either up-front
> migration or post-copy migration. In that light, it seems we won't be
> able to fully rid ourselves of the meta_bitmap idea, making the
> post-copy idea here not too useful in culling our complexity, since
> we'll have to support the current standard live migration anyway.
>
> So I have reviewed the current set of patches under the assumption that
> it seems like the right way to go for 2.4 and beyond.
>
> Thank you!
> --js

For now, post-copy migration is merged as I know. Is something changed 
for its reliability? Do we still need meta-bitmap approach for bitmap 
migration?

--
Best regards,
Vladimir
* now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.