Re: [Qemu-devel] qemu process crash: Assertion failed: QLIST_EMPTY(&bs->tracked_requests)

[Stefan Hajnoczi]

On Mon, Dec 11, 2017 at 01:42:38PM +0000, Fernando Casas Schössow wrote:
> Hello Stefan,
> 
> Thanks for your reply.
> Fortunately I didn’t have the problem again and it’s not clear how it can be 
> consistently reproduced. Daily backups are running as usual at the moment.
> 
> If there is anything I can do from my side or if you have any ideas to try to 
> reproduce it let me know.

Okay, thanks.

In case it happens again, here is some information about the assertion
failure.  It will probably be necessary to use GDB to inspect the failed
process (core dump).  The starting point for debugging is mirror_run()
in block/mirror.c:

  if (cnt == 0 && should_complete) {
      /* The dirty bitmap is not updated while operations are pending.
       * If we're about to exit, wait for pending operations before
       * calling bdrv_get_dirty_count(bs), or we may exit while the
       * source has dirty data to copy!
       *
       * Note that I/O can be submitted by the guest while
       * mirror_populate runs, so pause it now.  Before deciding
       * whether to switch to target check one last time if I/O has
       * come in the meanwhile, and if not flush the data to disk.
       */
      trace_mirror_before_drain(s, cnt);

      bdrv_drained_begin(bs);
      cnt = bdrv_get_dirty_count(s->dirty_bitmap);
      if (cnt > 0 || mirror_flush(s) < 0) {
          bdrv_drained_end(bs);
          continue;
      }

      /* The two disks are in sync.  Exit and report successful
       * completion.
       */
      assert(QLIST_EMPTY(&bs->tracked_requests));

The reason why this assertion makes sense is because
bdrv_drained_begin(bs) starts a region where I/O is quiesced (all
requests should have been completed).

Either there is a bug with blockjobs vs bdrv_drained_debug(), or maybe
mirror_flush() submitted new I/O requests.

It would be interesting to print the bs->tracked_requests linked list
in the GDB debugger.  Those requests might give a clue about the root
cause.

Stefan

signature.asc
Description: PGP signature