[Qemu-commits] [qemu/qemu] 86a3d5: block: Add .bdrv_co_block_status() callback

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 86a3d5c6889594b814d47a80e366aa4831676199
      
https://github.com/qemu/qemu/commit/86a3d5c6889594b814d47a80e366aa4831676199
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/io.c
    M include/block/block.h
    M include/block/block_int.h

  Log Message:
  -----------
  block: Add .bdrv_co_block_status() callback

We are gradually moving away from sector-based interfaces, towards
byte-based. Now that the block layer exposes byte-based allocation,
it's time to tackle the drivers.  Add a new callback that operates
on as small as byte boundaries. Subsequent patches will then update
individual drivers, then finally remove .bdrv_co_get_block_status().

The new code also passes through the 'want_zero' hint, which will
allow subsequent patches to further optimize callers that only care
about how much of the image is allocated (want_zero is false),
rather than full details about runs of zeroes and which offsets the
allocation actually maps to (want_zero is true).  As part of this
effort, fix another part of the documentation: the claim in commit
4c41cb4 that BDRV_BLOCK_ALLOCATED is short for 'DATA || ZERO' is a
lie at the block layer (see commit e88ae2264), even though it is
how the bit is computed from the driver layer.  After all, there
are intentionally cases where we return ZERO but not ALLOCATED at
the block layer, when we know that a read sees zero because the
backing file is too short.  Note that the driver interface is thus
slightly different than the public interface with regards to which
bits will be set, and what guarantees are provided on input.

We also add an assertion that any driver using the new callback will
make progress (the only time pnum will be 0 is if the block layer
already handled an out-of-bounds request, or if there is an error);
the old driver interface did not provide this guarantee, which
could lead to some inf-loops in drastic corner-case failures.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: e3efee828bc76e9780143f246fb0399eedd80c5e
      
https://github.com/qemu/qemu/commit/e3efee828bc76e9780143f246fb0399eedd80c5e
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/nvme.c

  Log Message:
  -----------
  nvme: Drop pointless .bdrv_co_get_block_status()

Commit bdd6a90 has a bug: drivers should never directly set
BDRV_BLOCK_ALLOCATED, but only io.c should do that (as needed).
Instead, drivers should report BDRV_BLOCK_DATA if it knows that
data comes from this BDS.

But let's look at the bigger picture: semantically, the nvme
driver is similar to the nbd, null, and raw drivers (no backing
file, all data comes from this BDS).  But while two of those
other drivers have to supply the callback (null because it can
special-case BDRV_BLOCK_ZERO, raw because it can special-case
a different offset), in this case the block layer defaults are
good enough without the callback at all (similar to nbd).

So, fix the bug by deletion ;)

Signed-off-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 3e4d0e72b77b70578d5530af588d6f0484e18325
      
https://github.com/qemu/qemu/commit/3e4d0e72b77b70578d5530af588d6f0484e18325
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/blkdebug.c
    M block/commit.c
    M block/io.c
    M block/mirror.c
    M block/throttle.c
    M include/block/block_int.h

  Log Message:
  -----------
  block: Switch passthrough drivers to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the generic helpers, and all passthrough clients
(blkdebug, commit, mirror, throttle) accordingly.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: a290f085901b528265787cd27ebda19c970be4ee
      
https://github.com/qemu/qemu/commit/a290f085901b528265787cd27ebda19c970be4ee
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/file-posix.c

  Log Message:
  -----------
  file-posix: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the file protocol driver accordingly.

In want_zero mode, we continue to report fine-grained hole
information (the caller wants as much mapping detail as possible);
but when not in that mode, the caller prefers larger *pnum and
merely cares about what offsets are allocated at this layer, rather
than where the holes live.  Since holes still read as zeroes at
this layer (rather than deferring to a backing layer), we can take
the shortcut of skipping lseek(), and merely state that all bytes
are allocated.

We can also drop redundant bounds checks that are already
guaranteed by the block layer.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 08c9e7735e4340a13596c5b97727655dee1fbb99
      
https://github.com/qemu/qemu/commit/08c9e7735e4340a13596c5b97727655dee1fbb99
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/gluster.c

  Log Message:
  -----------
  gluster: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the gluster driver accordingly.

In want_zero mode, we continue to report fine-grained hole
information (the caller wants as much mapping detail as possible);
but when not in that mode, the caller prefers larger *pnum and
merely cares about what offsets are allocated at this layer, rather
than where the holes live.  Since holes still read as zeroes at
this layer (rather than deferring to a backing layer), we can take
the shortcut of skipping find_allocation(), and merely state that
all bytes are allocated.

We can also drop redundant bounds checks that are already
guaranteed by the block layer.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: ba059e7b1785fbb7b3a60d4706f519b1cffab1e3
      
https://github.com/qemu/qemu/commit/ba059e7b1785fbb7b3a60d4706f519b1cffab1e3
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/iscsi.c

  Log Message:
  -----------
  iscsi: Switch cluster_sectors to byte-based

We are gradually converting to byte-based interfaces, as they are
easier to reason about than sector-based.  Convert all uses of
the cluster size in sectors, along with adding assertions that we
are not dividing by zero.

Improve some comment grammar while in the area.

Signed-off-by: Eric Blake <address@hidden>
Acked-by: Paolo Bonzini <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 04a408fbffa634bf84cb8f23e4f30c4eb8cb4b05
      
https://github.com/qemu/qemu/commit/04a408fbffa634bf84cb8f23e4f30c4eb8cb4b05
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/iscsi.c

  Log Message:
  -----------
  iscsi: Switch iscsi_allocmap_update() to byte-based

We are gradually converting to byte-based interfaces, as they are
easier to reason about than sector-based.  Convert all uses of
the allocmap (no semantic change).  Callers that already had bytes
available are simpler, and callers that now scale to bytes will be
easier to switch to byte-based in the future.

Signed-off-by: Eric Blake <address@hidden>
Acked-by: Paolo Bonzini <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 92809c36009de25d4f41e88f572bca3cb26b7387
      
https://github.com/qemu/qemu/commit/92809c36009de25d4f41e88f572bca3cb26b7387
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/iscsi.c

  Log Message:
  -----------
  iscsi: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the iscsi driver accordingly.  In this case,
it is handy to teach iscsi_co_block_status() to handle a NULL map
and file parameter, even though the block layer passes non-NULL
values, because we also call the function directly.  For now, there
are no optimizations done based on the want_zero flag.

We can also make the simplification of asserting that the block
layer passed in aligned values.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 05c33f1021f95bc18af153ea5669be57bba527e5
      
https://github.com/qemu/qemu/commit/05c33f1021f95bc18af153ea5669be57bba527e5
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/null.c

  Log Message:
  -----------
  null: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the null driver accordingly.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 8e0cf59d02d2e7b91e785214fbe9a15d8a6ce20b
      
https://github.com/qemu/qemu/commit/8e0cf59d02d2e7b91e785214fbe9a15d8a6ce20b
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/parallels.c

  Log Message:
  -----------
  parallels: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the parallels driver accordingly.  Note that
the internal function block_status() is still sector-based, because
it is still in use by other sector-based functions; but that's okay
because request_alignment is 512 as a result of those functions.
For now, no optimizations are added based on the mapping hint.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: d63b4c93e30915def5afabdb48134f5238ee793d
      
https://github.com/qemu/qemu/commit/d63b4c93e30915def5afabdb48134f5238ee793d
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/qcow.c

  Log Message:
  -----------
  qcow: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the qcow driver accordingly.  There is no
intent to optimize based on the want_zero flag for this format.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: a320fb04b66a2189b6cbd7924aa7bfaaded79afe
      
https://github.com/qemu/qemu/commit/a320fb04b66a2189b6cbd7924aa7bfaaded79afe
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the qcow2 driver accordingly.

For now, we are ignoring the 'want_zero' hint.  However, it should
be relatively straightforward to honor the hint as a way to return
larger *pnum values when we have consecutive clusters with the same
data/zero status but which differ only in having non-consecutive
mappings.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: b8d739fd6fb295a7562c506c65f2137199a509f9
      
https://github.com/qemu/qemu/commit/b8d739fd6fb295a7562c506c65f2137199a509f9
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/qed.c

  Log Message:
  -----------
  qed: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the qed driver accordingly, taking the opportunity
to inline qed_is_allocated_cb() into its lone caller (the callback
used to be important, until we switched qed to coroutines).  There is
no intent to optimize based on the want_zero flag for this format.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: d41aa7e36f442cba9c3841f87856454fca96288e
      
https://github.com/qemu/qemu/commit/d41aa7e36f442cba9c3841f87856454fca96288e
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/raw-format.c

  Log Message:
  -----------
  raw: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the raw driver accordingly.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 47943e986539e7e78ce4010f7d721408235ad058
      
https://github.com/qemu/qemu/commit/47943e986539e7e78ce4010f7d721408235ad058
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/sheepdog.c

  Log Message:
  -----------
  sheepdog: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the sheepdog driver accordingly.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Reviewed-by: Jeff Cody <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: b80666bf844bac5ae775cb67029a3cd67ad7c730
      
https://github.com/qemu/qemu/commit/b80666bf844bac5ae775cb67029a3cd67ad7c730
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/vdi.c

  Log Message:
  -----------
  vdi: Avoid bitrot of debugging code

Rework the debug define so that we always get -Wformat checking,
even when debugging is disabled.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Stefan Weil <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 67635f6abebac7f49d0067b3e60626c857cefa06
      
https://github.com/qemu/qemu/commit/67635f6abebac7f49d0067b3e60626c857cefa06
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/vdi.c

  Log Message:
  -----------
  vdi: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the vdi driver accordingly.  Note that the
TODO is already covered (the block layer guarantees bounds of its
requests), and that we can remove the now-unused s->block_sectors.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: c72080b9b8eab200551c26467511ebf599754f9e
      
https://github.com/qemu/qemu/commit/c72080b9b8eab200551c26467511ebf599754f9e
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/vmdk.c

  Log Message:
  -----------
  vmdk: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the vmdk driver accordingly.  Drop the
now-unused vmdk_find_index_in_cluster().

Also, fix a pre-existing bug: if find_extent() fails (unlikely,
since the block layer did a bounds check), then we must return a
failure, rather than 0.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 2f83673b57ba6b408222d5a18b48d34eb35a54b3
      
https://github.com/qemu/qemu/commit/2f83673b57ba6b408222d5a18b48d34eb35a54b3
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/vpc.c

  Log Message:
  -----------
  vpc: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the vpc driver accordingly.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: fba3998dae0b56f30aff3f3ad3a5cbc3502af0c6
      
https://github.com/qemu/qemu/commit/fba3998dae0b56f30aff3f3ad3a5cbc3502af0c6
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/vvfat.c

  Log Message:
  -----------
  vvfat: Switch to .bdrv_co_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Update the vvfat driver accordingly.  Note that we
can rely on the block driver having already clamped limits to our
block size, and simplify accordingly.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 636cb5125823a75ef6bbc9d63f8d7890576fdfec
      
https://github.com/qemu/qemu/commit/636cb5125823a75ef6bbc9d63f8d7890576fdfec
  Author: Eric Blake <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/io.c
    M include/block/block_int.h

  Log Message:
  -----------
  block: Drop unused .bdrv_co_get_block_status()

We are gradually moving away from sector-based interfaces, towards
byte-based.  Now that all drivers have been updated to provide the
byte-based .bdrv_co_block_status(), we can delete the sector-based
interface.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 18a59f03c36214105477e9bbaed5a99355718d27
      
https://github.com/qemu/qemu/commit/18a59f03c36214105477e9bbaed5a99355718d27
  Author: Anton Nefedov <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block: fix write with zero flag set and iovector provided

The normal bdrv_co_pwritev() use is either
  - BDRV_REQ_ZERO_WRITE clear and iovector provided
  - BDRV_REQ_ZERO_WRITE set and iovector == NULL

while
  - the flag clear and iovector == NULL is an assertion failure
    in bdrv_co_do_zero_pwritev()
  - the flag set and iovector provided is in fact allowed
    (the flag prevails and zeroes are written)

However the alignment logic does not support the latter case so the padding
areas get overwritten with zeroes.

Currently, general functions like bdrv_rw_co() do provide iovector
regardless of flags. So, keep it supported and use bdrv_co_do_zero_pwritev()
alignment for it which also makes the code a bit more obvious anyway.

Signed-off-by: Anton Nefedov <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: afe35cde6c2f3bdcee6e1f878836a64174fccb17
      
https://github.com/qemu/qemu/commit/afe35cde6c2f3bdcee6e1f878836a64174fccb17
  Author: Anton Nefedov <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M tests/qemu-iotests/033
    M tests/qemu-iotests/033.out

  Log Message:
  -----------
  iotest 033: add misaligned write-zeroes test via truncate

This new test case only makes sense for qcow2 while iotest 033 is generic;
however it matches the test purpose perfectly and also 033 contains those
do_test() tricks to pass the alignment, which won't look nice being
duplicated in other tests or moved to the common code.

Signed-off-by: Anton Nefedov <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 156b46ded3853dfc6b34c5afae019ff61798491b
      
https://github.com/qemu/qemu/commit/156b46ded3853dfc6b34c5afae019ff61798491b
  Author: Alberto Garcia <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M docs/interop/qcow2.txt

  Log Message:
  -----------
  specs/qcow2: Fix documentation of the compressed cluster descriptor

This patch fixes several mistakes in the documentation of the
compressed cluster descriptor:

1) the documentation claims that the cluster descriptor contains the
   number of sectors used to store the compressed data, but what it
   actually contains is the number of sectors *minus one* or, in other
   words, the number of additional sectors after the first one.

2) the width of the fields is incorrectly specified. The number of bits
   used by each field is

      x = 62 - (cluster_bits - 8)   for the offset field
      y = (cluster_bits - 8)        for the size field

   So the offset field's location is [0, x-1], not [0, x] as stated.

3) the size field does not contain the size of the compressed data,
   but rather the number of sectors where that data is stored. The
   compressed data starts at the exact point specified in the offset
   field and ends when there's enough data to produce a cluster of
   decompressed data. Both points can be in the middle of a sector,
   allowing several compressed clusters to be stored next to one
   another, sharing sectors if necessary.

Cc: address@hidden
Signed-off-by: Alberto Garcia <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: be820971ff6f88c156233eec8a50cdf8c70c0938
      
https://github.com/qemu/qemu/commit/be820971ff6f88c156233eec8a50cdf8c70c0938
  Author: Alberto Garcia <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M docs/qcow2-cache.txt

  Log Message:
  -----------
  docs: document how to use the l2-cache-entry-size parameter

This patch updates docs/qcow2-cache.txt explaining how to use the new
l2-cache-entry-size parameter.

Here's a more detailed technical description of this feature:

   https://lists.gnu.org/archive/html/qemu-block/2017-09/msg00635.html

And here are some performance numbers:

   https://lists.gnu.org/archive/html/qemu-block/2017-12/msg00507.html

Signed-off-by: Alberto Garcia <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: d2b63ba8dd20c1091b3f1033e6a95ef95b18149d
      
https://github.com/qemu/qemu/commit/d2b63ba8dd20c1091b3f1033e6a95ef95b18149d
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M include/block/aio.h
    M include/block/block.h

  Log Message:
  -----------
  aio: rename aio_context_in_iothread() to in_aio_context_home_thread()

The name aio_context_in_iothread() is misleading because it also returns
true when called on the main AioContext from the main loop thread, which
is not an IOThread.

This patch renames it to in_aio_context_home_thread() and expands the
doc comment to make the semantics clearer.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 7719f3c968c59e1bcda7e177679dc765b59e578f
      
https://github.com/qemu/qemu/commit/7719f3c968c59e1bcda7e177679dc765b59e578f
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block.c
    M block/io.c
    A include/block/aio-wait.h
    M include/block/block.h
    M include/block/block_int.h
    M util/Makefile.objs
    A util/aio-wait.c

  Log Message:
  -----------
  block: extract AIO_WAIT_WHILE() from BlockDriverState

BlockDriverState has the BDRV_POLL_WHILE() macro to wait on event loop
activity while a condition evaluates to true.  This is used to implement
synchronous operations where it acts as a condvar between the IOThread
running the operation and the main loop waiting for the operation.  It
can also be called from the thread that owns the AioContext and in that
case it's just a nested event loop.

BlockBackend needs this behavior but doesn't always have a
BlockDriverState it can use.  This patch extracts BDRV_POLL_WHILE() into
the AioWait abstraction, which can be used with AioContext and isn't
tied to BlockDriverState anymore.

This feature could be built directly into AioContext but then all users
would kick the event loop even if they signal different conditions.
Imagine an AioContext with many BlockDriverStates, each time a request
completes any waiter would wake up and re-check their condition.  It's
nicer to keep a separate AioWait object for each condition instead.

Please see "block/aio-wait.h" for details on the API.

The name AIO_WAIT_WHILE() avoids the confusion between AIO_POLL_WHILE()
and AioContext polling.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 33f2a7577787910bda161f428c904ac6a14b2454
      
https://github.com/qemu/qemu/commit/33f2a7577787910bda161f428c904ac6a14b2454
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block.c
    M block/block-backend.c

  Log Message:
  -----------
  block: add BlockBackend->in_flight counter

BlockBackend currently relies on BlockDriverState->in_flight to track
requests for blk_drain().  There is a corner case where
BlockDriverState->in_flight cannot be used though: blk->root can be NULL
when there is no medium.  This results in a segfault when the NULL
pointer is dereferenced.

Introduce a BlockBackend->in_flight counter for aio requests so it works
even when blk->root == NULL.

Based on a patch by Kevin Wolf <address@hidden>.

Signed-off-by: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: ad0df3e0fdac1d28e99ad29b99540d9f125e7ccf
      
https://github.com/qemu/qemu/commit/ad0df3e0fdac1d28e99ad29b99540d9f125e7ccf
  Author: Kevin Wolf <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M tests/Makefile.include
    A tests/test-block-backend.c

  Log Message:
  -----------
  block: test blk_aio_flush() with blk->root == NULL

This patch adds test cases for the scenario where blk_aio_flush() is
called on a BlockBackend with no root.  Calling drain afterwards should
complete the requests with -ENOMEDIUM.

Signed-off-by: Kevin Wolf <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 13471a40c13bc899c73f1879681da29da292547e
      
https://github.com/qemu/qemu/commit/13471a40c13bc899c73f1879681da29da292547e
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M hw/ide/core.c

  Log Message:
  -----------
  Revert "IDE: Do not flush empty CDROM drives"

This reverts commit 4da97120d51a4383aa96d741a2b837f8c4bbcd0b.

blk_aio_flush() now handles the blk->root == NULL case, so we no longer
need this workaround.

Cc: John Snow <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: efc75e2a4cf7dfa62c7ccaa9a1016f27e5519003
      
https://github.com/qemu/qemu/commit/efc75e2a4cf7dfa62c7ccaa9a1016f27e5519003
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block.c
    M block/crypto.c
    M block/file-posix.c
    M block/file-win32.c
    M block/gluster.c
    M block/iscsi.c
    M block/nfs.c
    M block/parallels.c
    M block/qcow.c
    M block/qcow2.c
    M block/qed.c
    M block/raw-format.c
    M block/rbd.c
    M block/sheepdog.c
    M block/ssh.c
    M block/vdi.c
    M block/vhdx.c
    M block/vmdk.c
    M block/vpc.c
    M include/block/block_int.h

  Log Message:
  -----------
  block: rename .bdrv_create() to .bdrv_co_create_opts()

BlockDriver->bdrv_create() has been called from coroutine context since
commit 5b7e1542cfa41a281af9629d31cef03704d976e6 ("block: make
bdrv_create adopt coroutine").

Make this explicit by renaming to .bdrv_co_create_opts() and add the
coroutine_fn annotation.  This makes it obvious to block driver authors
that they may yield, use CoMutex, or other coroutine_fn APIs.
bdrv_co_create is reserved for the QAPI-based version that Kevin is
working on.

Signed-off-by: Stefan Hajnoczi <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: c274393a3e69d101203cc0f9bfde6c64bbb9689b
      
https://github.com/qemu/qemu/commit/c274393a3e69d101203cc0f9bfde6c64bbb9689b
  Author: Stefan Hajnoczi <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: make qcow2_co_create2() a coroutine_fn

qcow2_create2() calls qemu_co_mutex_lock().  Only a coroutine_fn may
call another coroutine_fn.  In fact, qcow2_create2 is always called from
coroutine context.

Rename the function to add the "co" moniker and add coroutine_fn.

Reported-by: Marc-André Lureau <address@hidden>
Signed-off-by: Stefan Hajnoczi <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: be8fbd476393089f21e6ee9086d6d3dad8cb4f3d
      
https://github.com/qemu/qemu/commit/be8fbd476393089f21e6ee9086d6d3dad8cb4f3d
  Author: Max Reitz <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img: Make resize error message more general

The issue:

  $ qemu-img resize -f qcow2 foo.qcow2
  qemu-img: Expecting one image file name
  Try 'qemu-img --help' for more information

So we gave an image file name, but we omitted the length.  qemu-img
thinks the last argument is always the size and removes it immediately
from argv (by decrementing argc), and tries to verify that it is a valid
size only at a later point.

So we do not actually know whether that last argument we called "size"
is indeed a size or whether the user instead forgot to specify that size
but did give a file name.

Therefore, the error message should be more general.

Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1523458
Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 2b12a756ac2a5d3af5aa8116e3a3e62a1501ad61
      
https://github.com/qemu/qemu/commit/2b12a756ac2a5d3af5aa8116e3a3e62a1501ad61
  Author: Max Reitz <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/ssh.c

  Log Message:
  -----------
  block/ssh: Pull ssh_grow_file() from ssh_create()

If we ever want to offer even rudimentary truncation functionality for
ssh, we should put the respective code into a reusable function.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: bd8e0e32dac0cfb7c4e42b5a2d2b407819df049f
      
https://github.com/qemu/qemu/commit/bd8e0e32dac0cfb7c4e42b5a2d2b407819df049f
  Author: Max Reitz <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/ssh.c

  Log Message:
  -----------
  block/ssh: Make ssh_grow_file() blocking

At runtime (that is, during a future ssh_truncate()), the SSH session is
non-blocking.  However, ssh_truncate() (or rather, bdrv_truncate() in
general) is not a coroutine, so this resize operation needs to block.

For ssh_create(), that is fine, too; the session is never set to
non-blocking anyway.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 624f3006b8a26bcf7a0b2be13265ac99c65fd117
      
https://github.com/qemu/qemu/commit/624f3006b8a26bcf7a0b2be13265ac99c65fd117
  Author: Max Reitz <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/ssh.c

  Log Message:
  -----------
  block/ssh: Add basic .bdrv_truncate()

libssh2 does not seem to offer real truncation support, so we can only
grow files -- but that is better than nothing.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Richard W.M. Jones <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 9e029689e1228a218452d5c7b661ef208d22f71a
      
https://github.com/qemu/qemu/commit/9e029689e1228a218452d5c7b661ef208d22f71a
  Author: Alberto Garcia <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/qcow2-bitmap.c
    M block/qcow2-cluster.c
    M block/qcow2-refcount.c
    M block/qcow2-snapshot.c
    M block/qcow2.c
    M block/qcow2.h

  Log Message:
  -----------
  qcow2: Replace align_offset() with ROUND_UP()

The align_offset() function is equivalent to the ROUND_UP() macro so
there's no need to use the former. The ROUND_UP() name is also a bit
more explicit.

This patch uses ROUND_UP() instead of the slower QEMU_ALIGN_UP()
because align_offset() already requires that the second parameter is a
power of two.

Signed-off-by: Alberto Garcia <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 9d9b4b640f9e583ff4b24dc762630945f3ccc16d
      
https://github.com/qemu/qemu/commit/9d9b4b640f9e583ff4b24dc762630945f3ccc16d
  Author: Kevin Wolf <address@hidden>
  Date:   2018-03-02 (Fri, 02 Mar 2018)

  Changed paths:
    M block/qcow2-bitmap.c
    M block/qcow2-cluster.c
    M block/qcow2-refcount.c
    M block/qcow2-snapshot.c
    M block/qcow2.c
    M block/qcow2.h
    M block/ssh.c
    M qemu-img.c

  Log Message:
  -----------
  Merge remote-tracking branch 'mreitz/tags/pull-block-2018-03-02' into 
queue-block

Block patches

# gpg: Signature made Fri Mar  2 18:42:45 2018 CET
# gpg:                using RSA key F407DB0061D5CF40
# gpg: Good signature from "Max Reitz <address@hidden>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* mreitz/tags/pull-block-2018-03-02:
  qcow2: Replace align_offset() with ROUND_UP()
  block/ssh: Add basic .bdrv_truncate()
  block/ssh: Make ssh_grow_file() blocking
  block/ssh: Pull ssh_grow_file() from ssh_create()
  qemu-img: Make resize error message more general

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: bfe1a14c180ec44c033be12b9151252ffda69292
      
https://github.com/qemu/qemu/commit/bfe1a14c180ec44c033be12b9151252ffda69292
  Author: Kevin Wolf <address@hidden>
  Date:   2018-03-05 (Mon, 05 Mar 2018)

  Changed paths:
    M block/block-backend.c
    M qapi/block-core.json

  Log Message:
  -----------
  block: Fix NULL dereference on empty drive error

blk_error_action() sends a BLOCK_IO_ERROR QMP event which includes the
node name of its root node. If the BlockBackend represents an empty
drive, there is no root node, so we should not try to access its node
name. Make the field optional in the event and include it only when
the BlockBackend isn't empty.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>


  Commit: 58e2e17dba49b43f4ac9de19468aeae1c787dcc2
      
https://github.com/qemu/qemu/commit/58e2e17dba49b43f4ac9de19468aeae1c787dcc2
  Author: Peter Maydell <address@hidden>
  Date:   2018-03-06 (Tue, 06 Mar 2018)

  Changed paths:
    M block.c
    M block/blkdebug.c
    M block/block-backend.c
    M block/commit.c
    M block/crypto.c
    M block/file-posix.c
    M block/file-win32.c
    M block/gluster.c
    M block/io.c
    M block/iscsi.c
    M block/mirror.c
    M block/nfs.c
    M block/null.c
    M block/nvme.c
    M block/parallels.c
    M block/qcow.c
    M block/qcow2-bitmap.c
    M block/qcow2-cluster.c
    M block/qcow2-refcount.c
    M block/qcow2-snapshot.c
    M block/qcow2.c
    M block/qcow2.h
    M block/qed.c
    M block/raw-format.c
    M block/rbd.c
    M block/sheepdog.c
    M block/ssh.c
    M block/throttle.c
    M block/vdi.c
    M block/vhdx.c
    M block/vmdk.c
    M block/vpc.c
    M block/vvfat.c
    M docs/interop/qcow2.txt
    M docs/qcow2-cache.txt
    M hw/ide/core.c
    A include/block/aio-wait.h
    M include/block/aio.h
    M include/block/block.h
    M include/block/block_int.h
    M qapi/block-core.json
    M qemu-img.c
    M tests/Makefile.include
    M tests/qemu-iotests/033
    M tests/qemu-iotests/033.out
    A tests/test-block-backend.c
    M util/Makefile.objs
    A util/aio-wait.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Mon 05 Mar 2018 17:45:51 GMT
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <address@hidden>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (38 commits)
  block: Fix NULL dereference on empty drive error
  qcow2: Replace align_offset() with ROUND_UP()
  block/ssh: Add basic .bdrv_truncate()
  block/ssh: Make ssh_grow_file() blocking
  block/ssh: Pull ssh_grow_file() from ssh_create()
  qemu-img: Make resize error message more general
  qcow2: make qcow2_co_create2() a coroutine_fn
  block: rename .bdrv_create() to .bdrv_co_create_opts()
  Revert "IDE: Do not flush empty CDROM drives"
  block: test blk_aio_flush() with blk->root == NULL
  block: add BlockBackend->in_flight counter
  block: extract AIO_WAIT_WHILE() from BlockDriverState
  aio: rename aio_context_in_iothread() to in_aio_context_home_thread()
  docs: document how to use the l2-cache-entry-size parameter
  specs/qcow2: Fix documentation of the compressed cluster descriptor
  iotest 033: add misaligned write-zeroes test via truncate
  block: fix write with zero flag set and iovector provided
  block: Drop unused .bdrv_co_get_block_status()
  vvfat: Switch to .bdrv_co_block_status()
  vpc: Switch to .bdrv_co_block_status()
  ...

Signed-off-by: Peter Maydell <address@hidden>

# Conflicts:
#       include/block/block.h


Compare: https://github.com/qemu/qemu/compare/e1ee9ee139ed...58e2e17dba49