Re: query dirty areas according to bitmap via QMP or qemu-nbd

[Eric Blake]

On Fri, Jul 26, 2024 at 04:16:41PM GMT, Fiona Ebner wrote:
> Hi,
> 
> sorry if I'm missing the obvious, but is there a way to get the dirty
> areas according to a dirty bitmap via QMP? I mean as something like
> offset + size + dirty-flag triples. In my case, the bitmap is also
> exported via NBD, so same question for qemu-nbd being the client.

Over QMP, no - that can produce a potentially large response and
possible long time in computing the data, so we have never felt the
need to introduce a new QMP command for that purpose.  So over NBD is
the preferred solution.

> 
> I can get the info with "nbdinfo --map", but would like to avoid
> requiring a tool outside QEMU.

By default, QEMU as an NBD client only reads the "base:allocation" NBD
metacontext, and is not wired to read more than one NBD metacontext at
once (weaker than nbdinfo's capabilities).  But I have intentionally
left in a hack (accessible through QMP as well as from the command
line) for connecting a qemu NBD client to an alternative NBD
metacontext that feeds the block status, at which point 2 bits of
information from the alternative context are observable through the
result of block status calls.  Note that using such an NBD connection
for anything OTHER than block status calls is inadvisable (qemu might
incorrectly optimize reads based on its misinterpretation of those
block status bits); but as long as you limit the client to block
status calls, it's a great way to read out a "qemu:dirty-bitmap:..."
metacontext using only a qemu NBD client connection.

git grep -l x-dirty-bitmap tests/qemu-iotests

shows several of the iotests using the backdoor in just that manner.
In particular, tests/qemu-img-bitmaps gives the magic decoder ring:

| # x-dirty-bitmap is a hack for reading bitmaps; it abuses block status to
| # report "data":false for portions of the bitmap which are set
| IMG="driver=nbd,server.type=unix,server.path=$nbd_unix_socket"
| nbd_server_start_unix_socket -r -f qcow2 \
|     -B b0 -B b1 -B b2 -B b3 "$TEST_IMG"
| $QEMU_IMG map --output=json --image-opts \
|     "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b0" | _filter_qemu_img_map

meaning the map output includes "data":false for the dirty portions
and "data":true for the unchanged portions recorded in bitmap b0 as
read from the JSON map output.

> 
> If it is not currently possible, would upstream be interested too in the
> feature, either for QMP or qemu-nbd?

Improving qemu-img to get at the information without quite the hacky
post-processing deciphering would indeed be a useful patch, but it has
never risen to the level of enough of an itch for me to write it
myself (especially since 'nbdinfo --map's output works just as well).

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization:  qemu.org | libguestfs.org