[Qemu-commits] [qemu/qemu] f62492: block/qapi: Fix memory leak in qmp_query_blockstat...

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: f62492bb8d1ea7f7e156ffbdf411de46107072c5
      
https://github.com/qemu/qemu/commit/f62492bb8d1ea7f7e156ffbdf411de46107072c5
  Author: Kevin Wolf <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M block/qapi.c

  Log Message:
  -----------
  block/qapi: Fix memory leak in qmp_query_blockstats()

For BlockBackends that are skipped in query-blockstats, we would leak
info since commit 567dcb31. Allocate info only later to avoid the memory
leak.

Fixes: CID 1394727
Cc: address@hidden
Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>


  Commit: ef7a6a3c2a7725b169d054aa7487f9738bd6c4a6
      
https://github.com/qemu/qemu/commit/ef7a6a3c2a7725b169d054aa7487f9738bd6c4a6
  Author: Alberto Garcia <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M tests/qemu-iotests/093
    M tests/qemu-iotests/093.out

  Log Message:
  -----------
  qemu-iotests: Test removing a throttle group member with a pending timer

A throttle group can have several members, and each one of them can
have several pending requests in the queue.

The requests are processed in a round-robin fashion, so the algorithm
decides the drive that is going to run the next request and sets a
timer in it. Once the timer fires and the throttled request is run
then the next drive from the group is selected and a new timer is set.

If the user tried to remove a drive from a group and that drive had a
timer set then the code was not taking care of setting up a new timer
in one of the remaining members of the group, freezing their I/O.

This problem was fixed in 6fccbb475bc6effc313ee9481726a1748b6dae57,
and this patch adds a new test case that reproduces this exact
scenario.

Signed-off-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 5d8e4ca035f5a21e8634eb63a678bed55a1a94f9
      
https://github.com/qemu/qemu/commit/5d8e4ca035f5a21e8634eb63a678bed55a1a94f9
  Author: Alberto Garcia <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M block/throttle-groups.c

  Log Message:
  -----------
  throttle-groups: Skip the round-robin if a member is being drained

In the throttling code after an I/O request has been completed the
next one is selected from a different member using a round-robin
algorithm. This ensures that all members get a chance to finish their
pending I/O requests.

However, if a group member has its I/O limits disabled (because it's
being drained) then we should always give it priority in order to have
all its pending requests finished as soon as possible.

If we don't do this we could have a member in the process of being
drained waiting for the throttled requests of other members, for which
the I/O limits still apply.

This can have additional consequences: if we're running in qtest mode
(with QEMU_CLOCK_VIRTUAL) then timers can only fire if we advance the
clock manually, so attempting to drain a block device can hang QEMU in
the BDRV_POLL_WHILE() loop at the end of bdrv_do_drained_begin().

Signed-off-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 3db3e9c621519cbfc2b52e98b38f13ad863c0062
      
https://github.com/qemu/qemu/commit/3db3e9c621519cbfc2b52e98b38f13ad863c0062
  Author: Alberto Garcia <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M tests/qemu-iotests/093

  Log Message:
  -----------
  qemu-iotests: Update 093 to improve the draining test

The previous patch fixes a problem in which draining a block device
with more than one throttled request can make it wait first for the
completion of requests in other members of the same group.

This patch updates test_remove_group_member() in iotest 093 to
reproduce that scenario. This updated test would hang QEMU without the
fix from the previous patch.

Signed-off-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 25b8e4db7f3baca47244ccd945eb62b32a6e2019
      
https://github.com/qemu/qemu/commit/25b8e4db7f3baca47244ccd945eb62b32a6e2019
  Author: Alberto Garcia <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M block/throttle-groups.c

  Log Message:
  -----------
  throttle-groups: Don't allow timers without throttled requests

Commit 6fccbb475bc6effc313ee9481726a1748b6dae57 fixed a bug caused by
QEMU attempting to remove a throttle group member with no pending
requests but an active timer set. This was the result of a previous
bdrv_drained_begin() call processing the throttled requests but
leaving the timer untouched.

Although the commit does solve the problem, the situation shouldn't
happen in the first place. If we try to drain a throttle group member
which has a timer set, we should cancel the timer instead of ignoring
it.

Signed-off-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 497da8236ab2663a8108858ba7ea59aac21c5fe6
      
https://github.com/qemu/qemu/commit/497da8236ab2663a8108858ba7ea59aac21c5fe6
  Author: Fam Zheng <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M block/crypto.c

  Log Message:
  -----------
  luks: Allow share-rw=on

Format drivers such as qcow2 don't allow sharing the same image between
two QEMU instances in order to prevent image corruptions, because of
metadata cache. LUKS driver don't modify metadata except for when
creating image, so it is safe to relax the permission. This makes
share-rw=on property work on virtual devices.

Suggested-by: Daniel P. Berrangé <address@hidden>
Signed-off-by: Fam Zheng <address@hidden>
Reviewed-by: Daniel P. Berrangé <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: b24ec3c46281286a6a5624d0ceb5fa6f30bd8a4f
      
https://github.com/qemu/qemu/commit/b24ec3c46281286a6a5624d0ceb5fa6f30bd8a4f
  Author: Kevin Wolf <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M blockdev.c
    M hmp-commands.hx
    M hw/block/block.c
    M include/sysemu/blockdev.h
    M qemu-deprecated.texi
    M qemu-options.hx
    M tests/hd-geo-test.c

  Log Message:
  -----------
  block: Remove deprecated -drive geometry options

This reinstates commit a7aff6dd10b16b67e8b142d0c94c5d92c3fe88f6,
which was temporarily reverted for the 3.0 release so that libvirt gets
some extra time to update their command lines.

The -drive options cyls, heads, secs and trans were deprecated in
QEMU 2.10. It's time to remove them.

hd-geo-test tested both the old version with geometry options in -drive
and the new one with -device. Therefore the code using -drive doesn't
have to be replaced there, we just need to remove the -drive test cases.
This in turn allows some simplification of the code.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Markus Armbruster <address@hidden>


  Commit: 7f8fc971558219891ced425e8fe7a65d2a8f131f
      
https://github.com/qemu/qemu/commit/7f8fc971558219891ced425e8fe7a65d2a8f131f
  Author: Kevin Wolf <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M blockdev.c
    M device-hotplug.c
    M include/sysemu/blockdev.h
    M qemu-deprecated.texi
    M qemu-options.hx

  Log Message:
  -----------
  block: Remove deprecated -drive option addr

This reinstates commit eae3bd1eb7c6b105d30ec06008b3bc3dfc5f45bb,
which was temporarily reverted for the 3.0 release so that libvirt gets
some extra time to update their command lines.

The -drive option addr was deprecated in QEMU 2.10. It's time to remove
it.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Markus Armbruster <address@hidden>
Reviewed-by: Jeff Cody <address@hidden>


  Commit: 572023f7b235679716c4e57c9d701d83444f14b8
      
https://github.com/qemu/qemu/commit/572023f7b235679716c4e57c9d701d83444f14b8
  Author: Kevin Wolf <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M block/block-backend.c
    M blockdev.c
    M hw/block/block.c
    M hw/block/nvme.c
    M hw/block/virtio-blk.c
    M hw/ide/qdev.c
    M hw/scsi/scsi-disk.c
    M hw/usb/dev-storage.c
    M include/hw/block/block.h
    M include/sysemu/blockdev.h
    M qemu-deprecated.texi
    M qemu-options.hx
    M tests/ahci-test.c
    M tests/ide-test.c

  Log Message:
  -----------
  block: Remove deprecated -drive option serial

This reinstates commit b0083267444a5e0f28391f6c2831a539f878d424,
which was temporarily reverted for the 3.0 release so that libvirt gets
some extra time to update their command lines.

The -drive option serial was deprecated in QEMU 2.10. It's time to
remove it.

Tests need to be updated to set the serial number with -global instead
of using the -drive option.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Markus Armbruster <address@hidden>
Reviewed-by: Jeff Cody <address@hidden>


  Commit: 6984eb8da202ffb7ea91a473435cc1219b33a07b
      
https://github.com/qemu/qemu/commit/6984eb8da202ffb7ea91a473435cc1219b33a07b
  Author: Kevin Wolf <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M blockdev.c

  Log Message:
  -----------
  block: Remove dead deprecation warning code

This reinstates commit 6266e900b8083945cb766b45c124fb3c42932cb3,
which was temporarily reverted for the 3.0 release so that libvirt gets
some extra time to update their command lines.

We removed all options from the 'deprecated' array, so the code is dead
and can be removed as well.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Markus Armbruster <address@hidden>


  Commit: dbfdf6cb3625c68d0ea84ace4a786406cf43d4f3
      
https://github.com/qemu/qemu/commit/dbfdf6cb3625c68d0ea84ace4a786406cf43d4f3
  Author: Kevin Wolf <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M qapi/block-core.json

  Log Message:
  -----------
  qapi/block: Document restrictions for node names

blockdev-add fails if an invalid node name is given, so we should
document what a valid node name even is.

Reported-by: Cong Li <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Cong Li <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>


  Commit: 86fae10c64d642256cf019e6829929fa0d259c7a
      
https://github.com/qemu/qemu/commit/86fae10c64d642256cf019e6829929fa0d259c7a
  Author: Kevin Wolf <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M block/mirror.c
    M tests/qemu-iotests/041
    M tests/qemu-iotests/041.out

  Log Message:
  -----------
  mirror: Fail gracefully for source == target

blockdev-mirror with the same node for source and target segfaults
today: A node is in its own backing chain, so mirror_start_job() decides
that this is an active commit. When adding the intermediate nodes with
block_job_add_bdrv(), it starts the iteration through the subchain with
the backing file of source, though, so it never reaches target and
instead runs into NULL at the base.

While we could fix that by starting with source itself, there is no
point in allowing mirroring a node into itself and I wouldn't be
surprised if this caused more problems later.

So just check for this scenario and error out.

Cc: address@hidden
Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>


  Commit: 8d65a3ccfd5db7f0436e095cd952f5d0c3a873ba
      
https://github.com/qemu/qemu/commit/8d65a3ccfd5db7f0436e095cd952f5d0c3a873ba
  Author: Daniel P. Berrangé <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img: fix regression copying secrets during convert

When the convert command is creating an output file that needs
secrets, we need to ensure those secrets are passed to both the
blk_new_open and bdrv_create API calls.

This is done by qemu-img extracting all opts matching the name
suffix "key-secret". Unfortunately the code doing this was run after the
call to bdrv_create(), which meant the QemuOpts it was extracting
secrets from was now empty.

Previously this worked by luks as a bug meant the "key-secret"
parameters were not purged from the QemuOpts. This bug was fixed in

  commit b76b4f604521e59f857d6177bc55f6f2e41fd392
  Author: Kevin Wolf <address@hidden>
  Date:   Thu Jan 11 16:18:08 2018 +0100

    qcow2: Use visitor for options in qcow2_create()

Exposing the latent bug in qemu-img. This fix simply moves the copying
of secrets to before the bdrv_create() call.

Cc: address@hidden
Signed-off-by: Daniel P. Berrangé <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 3c005293c2ace58c796b917a792c34c02c4d6433
      
https://github.com/qemu/qemu/commit/3c005293c2ace58c796b917a792c34c02c4d6433
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M block.c
    M block/snapshot.c

  Log Message:
  -----------
  block: make .bdrv_close optional

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: f66b1f0e2795f6ac0646103a2b3e135985f3a80b
      
https://github.com/qemu/qemu/commit/f66b1f0e2795f6ac0646103a2b3e135985f3a80b
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M block/blkreplay.c
    M block/commit.c
    M block/copy-on-read.c
    M block/mirror.c
    M block/null.c
    M block/raw-format.c

  Log Message:
  -----------
  block: drop empty .bdrv_close handlers

.bdrv_close handler is optional after previous commit, no needs to keep
empty functions more.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 655b4b67e380891ee3996349213c62088d71395d
      
https://github.com/qemu/qemu/commit/655b4b67e380891ee3996349213c62088d71395d
  Author: Alberto Garcia <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M qobject/block-qdict.c

  Log Message:
  -----------
  qdict: Make qdict_extract_subqdict() accept dst = NULL

This function extracts all options from a QDict starting with a
certain prefix and puts them in a new QDict.

We'll have a couple of cases where we simply want to discard those
options instead of copying them, and that's what this patch does.

Signed-off-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 2f624b80ba67f19edb3f7c5bc27cddc3c844e443
      
https://github.com/qemu/qemu/commit/2f624b80ba67f19edb3f7c5bc27cddc3c844e443
  Author: Alberto Garcia <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: Remove children options from bs->{options,explicit_options}

When bdrv_open_inherit() opens a BlockDriverState the options QDict
can contain options for some of its children, passed in the form of
child-name.option=value

So while each child is opened with that subset of options, those same
options remain stored in the parent BDS, leaving (at least) two copies
of each one of them ("child-name.option=value" in the parent and
"option=value" in the child).

Having the children options stored in the parent is unnecessary and it
can easily lead to an inconsistent state:

  $ qemu-img create -f qcow2 hd0.qcow2 10M
  $ qemu-img create -f qcow2 -b hd0.qcow2 hd1.qcow2
  $ qemu-img create -f qcow2 -b hd1.qcow2 hd2.qcow2

  $ $QEMU -drive file=hd2.qcow2,node-name=hd2,backing.node-name=hd1

This opens a chain of images hd0 <- hd1 <- hd2. Now let's remove hd1
using block_stream:

  (qemu) block_stream hd2 0 hd0.qcow2

After this hd2 contains backing.node-name=hd1, which is no longer
correct because hd1 doesn't exist anymore.

This patch removes all children options from the parent dictionaries
at the end of bdrv_open_inherit() and bdrv_reopen_queue_child().

Signed-off-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 1bab38e7bd29347aca642c55a1de91ec6680efce
      
https://github.com/qemu/qemu/commit/1bab38e7bd29347aca642c55a1de91ec6680efce
  Author: Alberto Garcia <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: Simplify bdrv_reopen_abort()

If a bdrv_reopen_multiple() call fails, then the explicit_options
QDict has to be deleted for every entry in the reopen queue. This must
happen regardless of whether that entry's bdrv_reopen_prepare() call
succeeded or not.

This patch simplifies the cleanup code a bit.

Signed-off-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 4c8350fe1729b072473417dc22fa6410a4383127
      
https://github.com/qemu/qemu/commit/4c8350fe1729b072473417dc22fa6410a4383127
  Author: Alberto Garcia <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: Update bs->options if bdrv_reopen() succeeds

If bdrv_reopen() succeeds then bs->explicit_options is updated with
the new values, but bs->options never changes.

Here's an example:

   { "execute": "blockdev-add",
     "arguments": {
       "driver": "qcow2",
       "node-name": "hd0",
       "overlap-check": "all",
       "file": {
   "driver": "file",
   "filename": "hd0.qcow2"
       }
     }
   }

After this, both bs->options and bs->explicit_options contain
"overlap-check": "all".

Now let's change that using qemu-io's reopen command:

   (qemu) qemu-io hd0 "reopen -o overlap-check=none"

After this, bs->explicit_options contains the new value but
bs->options still keeps the old one.

This patch updates bs->options after a BDS has been successfully
reopened.

Signed-off-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 261dbcb18f80dad3720209285e2ddd4e1a0b22c3
      
https://github.com/qemu/qemu/commit/261dbcb18f80dad3720209285e2ddd4e1a0b22c3
  Author: Alberto Garcia <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: Simplify append_open_options()

This function returns a BDS's driver-specific options, excluding also
those from its children. Since we have just removed all children
options from bs->options there's no need to do this last step.

We allow references to children, though ("backing": "node0"), so those
we still have to remove.

Signed-off-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: b5fc2d306664c0c1c6c5cf8e164ffa7b8892283e
      
https://github.com/qemu/qemu/commit/b5fc2d306664c0c1c6c5cf8e164ffa7b8892283e
  Author: Peter Krempa <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M qapi/block-core.json

  Log Message:
  -----------
  qapi: block: Remove mentions of error types which were removed

Most of the various error classes were removed prior to the 1.2 release.
Remove mentions of the error classes which did not make it.

Signed-off-by: Peter Krempa <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: d3bd57d9f6a60187e381c6dbcb004701fb090be8
      
https://github.com/qemu/qemu/commit/d3bd57d9f6a60187e381c6dbcb004701fb090be8
  Author: Peter Maydell <address@hidden>
  Date:   2018-08-15 (Wed, 15 Aug 2018)

  Changed paths:
    M block.c
    M block/blkreplay.c
    M block/block-backend.c
    M block/commit.c
    M block/copy-on-read.c
    M block/crypto.c
    M block/mirror.c
    M block/null.c
    M block/qapi.c
    M block/raw-format.c
    M block/snapshot.c
    M block/throttle-groups.c
    M blockdev.c
    M device-hotplug.c
    M hmp-commands.hx
    M hw/block/block.c
    M hw/block/nvme.c
    M hw/block/virtio-blk.c
    M hw/ide/qdev.c
    M hw/scsi/scsi-disk.c
    M hw/usb/dev-storage.c
    M include/hw/block/block.h
    M include/sysemu/blockdev.h
    M qapi/block-core.json
    M qemu-deprecated.texi
    M qemu-img.c
    M qemu-options.hx
    M qobject/block-qdict.c
    M tests/ahci-test.c
    M tests/hd-geo-test.c
    M tests/ide-test.c
    M tests/qemu-iotests/041
    M tests/qemu-iotests/041.out
    M tests/qemu-iotests/093
    M tests/qemu-iotests/093.out

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Remove deprecated -drive options for geometry/serial/addr
- luks: Allow shared writers if the parents allow them (share-rw=on)
- qemu-img: Fix error when trying to convert to encrypted target image
- mirror: Fail gracefully for source == target
- I/O throttling: Fix behaviour during drain (always ignore the limits)
- bdrv_reopen() related fixes for bs->options/explicit_options content
- Documentation improvements

# gpg: Signature made Wed 15 Aug 2018 12:11:43 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <address@hidden>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (21 commits)
  qapi: block: Remove mentions of error types which were removed
  block: Simplify append_open_options()
  block: Update bs->options if bdrv_reopen() succeeds
  block: Simplify bdrv_reopen_abort()
  block: Remove children options from bs->{options,explicit_options}
  qdict: Make qdict_extract_subqdict() accept dst = NULL
  block: drop empty .bdrv_close handlers
  block: make .bdrv_close optional
  qemu-img: fix regression copying secrets during convert
  mirror: Fail gracefully for source == target
  qapi/block: Document restrictions for node names
  block: Remove dead deprecation warning code
  block: Remove deprecated -drive option serial
  block: Remove deprecated -drive option addr
  block: Remove deprecated -drive geometry options
  luks: Allow share-rw=on
  throttle-groups: Don't allow timers without throttled requests
  qemu-iotests: Update 093 to improve the draining test
  throttle-groups: Skip the round-robin if a member is being drained
  qemu-iotests: Test removing a throttle group member with a pending timer
  ...

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/c146b54c7fdb...d3bd57d9f6a6
      **NOTE:** This service has been marked for deprecation: 
https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.