Re: [Qemu-devel] live block copy/stream/snapshot discussion

[Marcelo Tosatti]

On Mon, Jul 11, 2011 at 03:47:15PM +0100, Stefan Hajnoczi wrote:
> Kevin, Marcelo,
> I'd like to reach agreement on the QMP/HMP APIs for live block copy
> and image streaming.  Libvirt has acked the image streaming APIs that
> Adam proposed and I think they are a good fit for the feature.  I have
> described that API below for your review (it's exactly what the QED
> Image Streaming patches provide).
> 
> Marcelo: Are you happy with this API for live block copy?  Also please
> take a look at the switch command that I am proposing.
> 
> Image streaming API
> ===================
> 
> For leaf images with copy-on-read semantics, the stream commands allow the 
> user
> to populate local blocks by manually streaming them from the backing image.
> Once all blocks have been streamed, the dependency on the original backing
> image can be removed.  Therefore, stream commands can be used to implement
> post-copy live block migration and rapid deployment.
> 
> The block_stream command can be used to stream a single cluster, to
> start streaming the entire device, and to cancel an active stream.  It
> is easiest to allow the block_stream command to manage streaming for the
> entire device but a managent tool could use single cluster mode to
> throttle the I/O rate.
> 
> The command synopses are as follows:
> 
> block_stream
> ------------
> 
> Copy data from a backing file into a block device.
> 
> If the optional 'all' argument is true, this operation is performed in the
> background until the entire backing file has been copied.  The status of
> ongoing block_stream operations can be checked with query-block-stream.
> 
> Arguments:
> 
> - all:    copy entire device (json-bool, optional)
> - stop:   stop copying to device (json-bool, optional)
> - device: device name (json-string)

It must be possible to specify backing file that will be
active after streaming finishes (data from that file will not 
be streamed into active file, of course).

> Return:
> 
> - device: device name (json-string)
> - len:    size of the device, in bytes (json-int)
> - offset: ending offset of the completed I/O, in bytes (json-int)
> 
> Examples:
> 
> -> { "execute": "block_stream", "arguments": { "device": "virtio0" } }
> <- { "return":  { "device": "virtio0", "len": 10737418240, "offset": 512 } }
> 
> -> { "execute": "block_stream", "arguments": { "all": true, "device":
> "virtio0" } }
> <- { "return": {} }
> 
> -> { "execute": "block_stream", "arguments": { "stop": true, "device":
> "virtio0" } }
> <- { "return": {} }
> 
> query-block-stream
> ------------------
> 
> Show progress of ongoing block_stream operations.
> 
> Return a json-array of all operations.  If no operation is active then an 
> empty
> array will be returned.  Each operation is a json-object with the following
> data:
> 
> - device: device name (json-string)
> - len:    size of the device, in bytes (json-int)
> - offset: ending offset of the completed I/O, in bytes (json-int)
> 
> Example:
> 
> -> { "execute": "query-block-stream" }
> <- { "return":[
>        { "device": "virtio0", "len": 10737418240, "offset": 709632}
>     ]
>   }
> 
> 
> Block device switching API
> ==========================
> 
> Extend the 'change' command to support changing the image file without
> media change notification.
> 
> Perhaps we should take the opportunity to add a "format" argument for
> image files?
> 
> change
> ------
> 
> Change a removable medium or VNC configuration.
> 
> Arguments:
> 
> - "device": device name (json-string)
> - "target": filename or item (json-string)
> - "arg": additional argument (json-string, optional)
> - "notify": whether to notify guest, defaults to true (json-bool, optional)
> 
> Examples:
> 
> 1. Change a removable medium
> 
> -> { "execute": "change",
>              "arguments": { "device": "ide1-cd0",
>                             "target": "/srv/images/Fedora-12-x86_64-DVD.iso" 
> } }
> <- { "return": {} }
> 
> 2. Change a disk without media change notification
> 
> -> { "execute": "change",
>              "arguments": { "device": "virtio-blk0",
>                             "target": "/srv/images/vm_1.img",
>                             "notify": false } }
> 
> 3. Change VNC password
> 
> -> { "execute": "change",
>              "arguments": { "device": "vnc", "target": "password",
>                             "arg": "foobar1" } }
> <- { "return": {} }
> 
> How live block copy works
> =========================
> 
> Live block copy does the following:
> 
> 1. Create the destination file: qemu-img create -f $cow_fmt -o
> backing_file=$base destination.$cow_fmt
> 2. Switch to the destination file: change -n virtio-blk0 /srv/images/vm_1.img

The snapshot command (snapshot_blkdev) can be used for these two steps.

> 3. Stream the base into the image file: block_stream -a virtio-blk0
> 
> Stefan