Re: [Qemu-devel] vm state save/restore question

[Benjamin Herrenschmidt]

On Tue, 2012-06-19 at 16:59 +0200, Juan Quintela wrote:
> >> - The hash table (mentioned above). This is just a big chunk of
> memory
> >> (it will routinely be 16M), so I really don't want to start
> iterating
> >> all elements, just a bulk load will do, and the size might actually
> be
> >> variable.
> 
> This is going to kill migration download time.  With current setup, we
> just sent something like 1-2MB in stage 3 (i.e. after the machine is
> down).  Default downtime is 30ms, And 16MB is going to take around 1s
> on gigabit ethenet.
> 
> Once said that, if you told me the state that you want to sent, I can
> take a look.

Well, we don't have much of a choice unless we do something really fancy
but that would be a second step...

The MMU hash table on power is where all our translations go. What we
could do is put in some knowledge about what translations are actually
necessary for the guest and which ones can be rebuild (faulted in),
essentially by adding knowledge to qemu/kvm about the "bolted" bit that
the guests uses for translations that must not be evicted.

However, that would require at least some interaction between the guest
and qemu/kvm to enable the function since this bit is a guest SW
construct (unless it got architected in recent PAPR, I need to dbl
check).

In that case, we save a "compressed" variant of the hash table (RLE
style ? we need to keep the hash location of each entry ...) which
contains only the bolted translations.

The consequence is that post-migration, the guest will temporarily slow
down as most translations will have to be faulted back in.

> >> - The XICS (interrupt controller). The actual size of the interrupt
> >> state array can vary (the number of interrupt sources can vary,
> it's
> >> fixed today by the machine code but I wouldn't rely too much on
> that and
> >> in any case, from the XICS driver perspective, it's not a constant,
> it's
> >> a variable it gets passed when initializing).
> 
> Can you point me at the structure that you want to sent?

hw/xics.c, there's several bits here but the "array" itself is
the struct ics_state which has a pointer to an array of  struct
ics_irq_state.

However, I think I figured out how to do that by using the array macro
which takes the size of the array as a pointer to a variable containing
that size or am I mistaken ?

> >> So in both these cases, I need either code to control the save/load
> >> process (old style ? hard to hook into vmstate as far as I can
> tell) or
> >> maybe a way to describe the array so that the array size itself is
> a
> >> pointer to a variable (Andreas mentioned something along those
> lines).
> >> Is there any doco for that stuff btw ? I haven't seen anything
> >> detailed...
> >
> > I'm sure Juan knows more there :)
> 
> thanks for pointing me to the discussion O:-)

Thanks for the help !

Cheers,
Ben.