Re: [Qemu-devel] [PATCH v9 00/56] Postcopy implementation

[Dr. David Alan Gilbert]

* Bharata B Rao (address@hidden) wrote:
> On Fri, Nov 06, 2015 at 03:48:11PM +0000, Dr. David Alan Gilbert wrote:
> > * Bharata B Rao (address@hidden) wrote:
> > 
> > > > Where we have iterable, but non-postcopiable devices (e.g. htab
> > > > or block migration), complete them before forming the 'package'
> > > > but with the CPUs stopped.  This stops them filling up the package.
> > > 
> > > That helps and the migration suceeds now when I switch to postcopy
> > > immediately after starting the migration.
> > 
> > Excellent.
> > 
> > > However after postcopy migration, when I attempt to start an incoming
> > > instance again to migrate the guest back, I see this failure:
> > > 
> > > qemu-system-ppc64: cannot set up guest memory 'ppc_spapr.ram': Cannot 
> > > allocate memory
> > > 
> > > The same doesn't happen with normal migration.
> > 
> > Huh that's fun; that's the original source guest that's running out of RAM?
> > It's original QEMU should be gone by that point.
> 
> Yes, the original source QEMU is gone, but there is not enough memory left
> in the host to start another incoming QEMU instance because...

Ah, so this is ping-pong on one host?

> At the beginning
> -----------------
> $ grep -i mem /proc/meminfo 
> MemTotal:       132816832 kB
> MemFree:        128781632 kB
> MemAvailable:   131668224 kB
> 
> After starting the guest (-m 64G,slots=32,maxmem=128G)
> ------------------------
> $ grep -i mem /proc/meminfo 
> MemTotal:       132816832 kB
> MemFree:        124866880 kB
> MemAvailable:   127753728 kB
> 
> After starting the destination instance (incoming)
> -------------------------------------------------
> $ grep -i mem /proc/meminfo 
> MemTotal:       132816832 kB
> MemFree:        122514880 kB
> MemAvailable:   125401920 kB
> 
> After postcopy migration completes
> ----------------------------------
> $ grep -i mem /proc/meminfo 
> MemTotal:       132816832 kB
> MemFree:        55150592 kB
> MemAvailable:   58037888 kB
> 
> After terminating the source instance
> -------------------------------------
> $ grep -i mem /proc/meminfo 
> MemTotal:       132816832 kB
> MemFree:        59432448 kB
> MemAvailable:   62319872 kB
> 
> So as you can see, postcopy migration will result in guest claiming
> its entire RAM memory from host. This doesn't happen during normal migration.

I'll try and see if I can replicate this.

Can you :
   1) show me the command line you're using
   2) Show me /proc/pid/smaps for the destination qemu
   3) Turn on the trace     postcopy_place_page_zero
      the theory is that most of your pages should end up as zero pages
      and not be allocated.

Dave

> The above results are with the additional patch you sent in this thread
> and I was switching to postcopy migration immediately after starting the
> migration. Without this patch, when I delay the switching to postcopy
> migration a bit, I see the free memory as below when postcopy migration
> completes.
> 
> $ grep -i mem /proc/meminfo 
> MemTotal:       132816832 kB
> MemFree:        85018176 kB
> MemAvailable:   87937024 kB
> 
> Regards,
> Bharata.
> 
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK