Re: [Qemu-devel] [RFC] qmp interface for save vmstate to image

[Wenchao Xia]

Hi, Pavel
  Sorry for late response.
> Hi Wenchao,
> 
> It seems the we are working on the same thing. You are trying to improve
> the size of vmstate if you want to save it to file or as an internal
> snapshot.
> 
> I'm also working on that issue and I think that my solution could be
> also used for savevm to external file or for live backup.
> 
> Here is my proposal how to do it:
> 
> We will not have the fixed size of vmstate, we will have the possible
> minimal size of the vmstate. I will also use the migration code to save
> the vmstate.
> 
  It is good if speed and size can be improved, but IMHO the size will
be a problem. Predictable or fixed size ensure management stack to
give assess and decision, preserve resource ahead, personally I
does not like a process continue to take resource without limit,
in most case I'll turn it off.... By using qcow2m vmstate will have a
fixed MAX size, ideal to be used to take it as a backup data.
  Above is my personal opinion, and I do want to know the maintainer's
opinion to decide whether to continue.

> In the qemu_savevm_state_begin we will create bitmap for all ram pages.
> Then we set all pages in bitmap to "1" and it means we need to save them
> all. Then we check all ram pages for duplicated pages and we will unset
> all duplicated pages from "savevm_bitmap".
> 
> In the qemu_savevm_state_iterate we will start saving remaining ram
> pages according to "savevm_bitmap". Because the guest is running, it
> could change the data in ram pages which is still not saved. For this
> case we also have to create a priority queue. Into this priority queue
> we will copy every ram page before it will be changed and also remove
> this ram page from savevm_bitmap. In the iterate cycle we will at first
> handle the priority queue and then continue to other ram pages from the
> savevm_bitmap.
> 
  OK, I got your idea: intercept the page writing before it changes.
I think this could reduce time in savevm. But some problems need to be
confirmed:
1 is it workable when KVM is used? In my understanding KVM will directly
change the ram page before qemu can take over.
2 the performance sacrifice of running guest, need a test.
3 the total buffer size in the queue. If you plan to make it used for
any migration then in TCP case the buffer may grow to a large size
for speed reason. If you use it only for local device, I suggest
conclude it as a improvement for migrate to block device, in contrast
to migrate to stream, then the performance optimizing infra such
as buffer/cache can be used much easier, to reduce the performance
lost in page changing.
  I feel this is more likely as an algorithm improvement for block
migration, which can work with my patch together. My patch
is actually introducing migrate vmstate to block instead of stream.

> In the qemu_savevm_state_complete we will save only non-live data.
> 
> This should reduce the vmstate size and also speedup the saving of
> vmstate with minimal memory usage.
> 
> Pavel
> 
> On 03/15/2013 08:24 AM, Wenchao Xia wrote:
>> Hi, Juan and guys,
>>     I'd like to add a new way to save vmstate, which will based on the
>> migration thread, but will write contents to block images, instead
>> of fd as stream. Following is the method to add API:
>>
>> 1 add parameters to migrate interface, and a new type of uri:
>> image:[VMSATE_SAVE_IMAGE]
>>
>> ##
>> # @MigrateImageOptions:
>> #
>> # Options for migration to image.
>> #
>> # @path: the full path to the image to be used.
>> # @use-existing: #optional, whether to use existing image in path. If
>> #                not specified, qemu will try create new image.
>> # @create-size: #optional, the image's virtual size in creation. Only
>> #               valid when use-existing is false or absence, unit is M.
>> # @fmt: #optional the format of the image. If not specified, when
>> #       use-existing is true, qemu will try detect the image format,
>> #       when use-existing is false or absence, qcow2 format will be
>> #       used.
>> # @stream: #optional, whether to save vmstate as stream, in which way
>> #          small writes reduce but size may continue growing. If not
>> #          specified, vmstate will be saved with fixed size.
>> #
>> # Since: 1.5
>> ##
>> { 'type': 'MigrateImageOptions',
>>     'data': { 'path': 'str', '*use-existing': 'bool',
>>               '*create-size': 'int', '*fmt': 'str',
>>               '*stream': 'bool' } }
>>
>> ##
>> # @migrate
>> #
>> # Migrates the current running guest to another Virtual Machine.
>> #
>> # @uri: the Uniform Resource Identifier of the destination VM
>> #
>> # @blk: #optional do block migration (full disk copy)
>> #
>> # @inc: #optional incremental disk copy migration
>> #
>> # @detach: this argument exists only for compatibility reasons and
>> #          is ignored by QEMU
>> #
>> # @image-options: #optional, the options used in migration to image.
>> #                 Only valid in migration to image.
>> #
>> # Returns: nothing on success
>> #
>> # Since: 0.14.0
>> ##
>> { 'command': 'migrate',
>>     'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool',
>>              '*detach': 'bool', '*image-options': MigrateImageOptions} }
>>
>>     In this way query-migrate and migrate incoming could be naturelly used
>> for querying and restoring, But introduce some options only for the
>> image migration.
>>
>> 2 new command vmstate-save with above options. Then use query-migrate
>> and migrate incoming to query/restore the states, which seems wild.
>>
>>     I can't decide which is better, could u take a look and put some
>> comments on this?
>>
> 


-- 
Best Regards

Wenchao Xia