Re: [Qemu-devel] Live migration hangs after migration to remote host

[Dr. David Alan Gilbert]

* Eduardo Otubo (address@hidden) wrote:
> On Wed, Jul 29, 2015 at 09=11=21AM +0100, Dr. David Alan Gilbert wrote:
> > * Eduardo Otubo (address@hidden) wrote:
> > > On Tue, Jul 28, 2015 at 04=19=46PM +0100, Dr. David Alan Gilbert wrote:
> > > > * Eduardo Otubo (address@hidden) wrote:
> > > > > Hello all,
> > > > > 
> > > > > I'm facing a weird behavior on my tests: I am able to live migrate
> > > > > between two virtual machines on my localhost, but not to another
> > > > > machine, both using tcp.
> > > > > 
> > > > > * I am using the same arguments on the command line;
> > > > > * Both virtual machines uses the same qcow2 file visible through NFS;
> > > > > * Both machines are in the same subnet;
> > > > > * Migration is being done from intel to intel;
> > > > > * Same version of Qemu (github master - f8787f8723);
> > > > > 
> > > > > Using all above I am able to live migrate on the same host: between 
> > > > > two
> > > > > vms on local host or between two vms in the remote host; but when
> > > > > migrating from local to remote, the guest hangs. I still can access 
> > > > > its
> > > > > console via ctrl+alt+2, though, and everything seems to be normal. If 
> > > > > I
> > > > > issue a reboote via console on the remote, the guest gets back to
> > > > > normal.
> > > > > 
> > > > > Am I missing something here?
> > > > 
> > > > Just checking, but are you saying that as far as qemu is concerned, the 
> > > > migration
> > > > is happy, it's just the guest that's hung?
> > > 
> > > That's exactly the case. The console (via ctrl+alt+2) is active and
> > > responding to all commands normally, but the screen (ctrl+alt+1) is
> > > frozen and I can't interact with it at all.
> > 
> > Are you driving this via libvirt or using qemu monitor directly?
> > If the latter, can you please get an 'info migrate' from the source
> > and an 'info status' from the destination at the end of migrate.
> 
> I'm using qemu command line directly. And I got the problem :) See
> below.
> 
> > 
> > > > Are the host clocks on the two hosts very close (there are lots of
> > > > weird corner cases with mismatched clocks) - same time zone?
> > > 
> > > Yep. Both machines are in the same room and have the clock sync'ed.
> > 
> > OK, good.
> > 
> > > > 
> > > > Are you using cache=none (given that it's NFS shared)
> > > 
> > > I wasn't. But I tried again with cache=none and I got exactly the same
> > > thing.
> > 
> > OK, and this pair of machines, have you tried both directions - i.e.
> > going a->b and b->a - do both directions fail?
> > Is the NFS server one of the two machines?  If it is, and you're using 
> > libvirt,
> > make sure that the directory the disks are on is an NFS mount on both
> > machines; e.g. don't migrate directly from the NFS export.
> > 
> > > Also, I tried with stable-2.2 branch and got the same behavior. I really
> > > think that's very unlikely to have unstable code of such an important
> > > feature upstream, or on a stable- branch. Most probable thing is that
> > > I have something wrong on my environment.
> > 
> > Yes, the challenge is to find what; and if it's something common
> > we should try and find a way of spotting it.
> > 
> > > Anyway, I'll keep tetsing different stable- branches until I find
> > > something that works for me. I'll keep the mailing list posted.
> > 
> > Could you share the qemu command line so we can see if we can
> > spot anything?
> 
> Got the problem! I tried to simplify my qemu command line to the
> smallest possible, excluding things I thought it could cause the issue.
> With no further due, this is the argument:
> 
>     -cpu 'Opteron_G4'
> 
> Without this argument everything works as it should, console responsive
> and guest active :)

Can you show cat /proc/cpuinfo off the two hosts?
(Only one CPU, but please include the whole entry)

Dave

> It says on the documentation[1] that it's possible to migrate between
> AMD and Intel, but I think I got a corner case. Apparently I can't
> specify the exact CPU model. Is this a known issue? Couldn't find any
> reference on bugzilla or launchpad.
> 
> [1] - http://www.linux-kvm.org/page/Migration
> 
> -- 
> Eduardo Otubo
> ProfitBricks GmbH


--
Dr. David Alan Gilbert / address@hidden / Manchester, UK