[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] While testing `drive-backup` with 'dirty-bitmap' syc mo
From: |
John Snow |
Subject: |
Re: [Qemu-devel] While testing `drive-backup` with 'dirty-bitmap' syc mode via QMP. . . |
Date: |
Tue, 21 Apr 2015 11:34:07 -0400 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 |
Kashyap Chamarthy sent me a question asking for some clarifications on
testing my incremental backup series. I responded off-list but he has
asked me to reproduce my answer for the archives :)
On 04/21/2015 08:35 AM, Kashyap Chamarthy wrote:
Hi John,
I would have asked this question on #qemu on OFTC, but there are three
to four parallel conversations ongoing at the moment. . .
So, I'm testing on a branch with your "v2.5" tree from
https://github.com/jnsnow/qemu/commits/incremental-transactions
$ git checkout -b v2.5-and-v6-inc-backup jsnow/incremental-transactions
$ git describe
v2.3.0-rc3-37-g6932a32
Test
----
(1) I have the QEMU invocation w/ QMP of a CirrOS disk image, that way:
/home/kashyapc/tinker-space/qemu-upstream/x86_64-softmmu/qemu-system-x86_64 \
. . .
-drive file=/export/full_backup.img,if=none,id=drive-ide0-0-0,format=qcow2 \
-qmp tcp:localhost:4444,server
So, it's waiting on the server:
$ ./2-min-qemu-invoc-qmp.sh
char device redirected to /dev/pts/31 (label charserial0)
QEMU waiting for connection on: disconnected:tcp:localhost:4444,server
(2) Then, when I execute the `drive-backup` QMP command:
-----------------------------------
#!/bin/bash
set -x
# Test incremental backup with `drive-backup`
exec 3<>/dev/tcp/localhost/4444
echo -e "{ 'execute': 'qmp_capabilities' }" >&3
read response <&3
echo $response
echo -e "{ 'execute': 'drive-backup', 'arguments':
{ 'device': 'drive-ide0-0-0', 'bitmap': 'bitmap0', 'sync':
'dirty-bitmap', 'target':
'/home/kashyapc/work/virt/qemu/incremental-backup-test-qemu/tests/incremental.0.img',
'mode': 'existing', 'format': 'qcow2' } }" >&3
read response <&3
echo $response
-----------------------------------
(3) Invoke it:
$ ./invoke-drive-backup.sh
{"QMP": {"version": {"qemu": {"micro": 93, "minor": 2, "major": 2}, "package": ""},
"capabilities": []}}
{"return": {}}
Now, I don't see any feedback notification on the QMP server as a result
of the `drive-bacup` execution from step (3) unlike from a slightly old
test of regular `drive-backup` command[1].
NOTE: I tested with and without pre-creating the 'incremental.0.img'
file, to no avail.
Am I missing anything obvious?
[1] https://kashyapc.fedorapeople.org/virt/test-qmp-drive-backup.txt
(1) It doesn't appear as though you've created a bitmap in this example?
(2) Incremental backup is designed to work basically exclusively with
'mode': 'existing' because it relies on the user (or the management
utility/api) to keep track of what the "last backup" was.
(I had actually attempted to add that convenience feature into QEMU, but
the sense was that the complexities of that were best left to libvirt,
and I agreed.)
Here's the workflow (if the VM has not begun guest execution)
(A) Create a bitmap ("bitmap0" is my usual choice)
(B) Create an "anchor" or "tail" backup:
This is a full backup of the drive, performed as usual.
(C) Create the image to be used, using e.g. tail.qcow2 as
the backing file.
(D) Ensure that <bitmap0> is clear by issuing a clear command. Look
at bitmaps.md for an example.
(E) You may optionally start the VM to allow the disk to change
at this point.
(F) You may now issue an incremental backup command:
{ 'execute': 'drive-backup',
'arguments': {
'device': 'drive-ide0-0-0',
'bitmap': 'bitmap0',
'sync': 'dirty-bitmap',
'target': .../tail.qcow2,
'mode': 'existing',
'format': 'qcow2'
}
}
The workflow is different if the VM has already started,
for data safety reasons. (We need extra precautions to ensure
the coherency of the incremental backup.)
(A) Create a bitmap. ("bitmap0")
(B) Using a single transaction, you must accomplish the following:
(i) Create a full backup of 'drive-ide0-0-0'
(ii) Clear bitmap0.
This ensures that the bitmap tracks only writes that have
occurred after the full backup was made. This is why the
transactions are important.
It's OK to let QEMU create the tail.qcow2 file here,
so we don't need mode:existing.
{ 'execute': 'transaction',
'arguments': {
'actions': [
{ 'type': 'block-dirty-bitmap-clear',
'data': {
"node": 'drive-ide0-0-0',
"name": 'bitmap0'
}
},
{ 'type': 'drive-backup',
'data': {
'device': 'drive-ide0-0-0',
'bitmap': 'bitmap0',
'sync': 'full',
'target': .../tail.qcow2,
'format': 'qcow2'
}
}
]
}
}
(C) We need to create the incremental backup image outside of QEMU;
# qemu-img create -f qcow2 -F qcow2 -b .../tail.qcow2
(D) Now that we have a full backup and a synced bitmap, we can
issue a standard incremental backup command at any time.
{ 'execute': 'drive-backup',
'arguments': {
'device': 'drive-ide0-0-0',
'bitmap': 'bitmap0',
'sync': 'dirty-bitmap',
'target': .../tail.qcow2,
'mode': 'existing',
'format': 'qcow2'
}
}
(3) You are probably not seeing any return from this script because it
looks as if you only read from the QMP stream once, which will just give
you the reply from accepting the QMP command, but the event that tells
you if it completed successfully or not occurs later, so you need one
more read. If you use bash to send/receive QMP commands, you need to
poll the fd to get event responses that occur later.
--
—js
- Re: [Qemu-devel] While testing `drive-backup` with 'dirty-bitmap' syc mode via QMP. . .,
John Snow <=