Re: migration qtest failure: "query-migrate shows failed migration: Unable to write to socket: Broken pipe"

[Peter Maydell]

On Thu, 15 Dec 2022 at 11:40, Dr. David Alan Gilbert
<dgilbert@redhat.com> wrote:
>
> * Peter Maydell (peter.maydell@linaro.org) wrote:
> > Hi; I see this migration qtest failure on my x86 macos box:
> >
> >
> > ▶  32/591 
> > ERROR:../../tests/qtest/migration-helpers.c:151:migrate_query_not_failed:
> > assertion fail
> > ed: (!g_str_equal(status, "failed")) ERROR
> >  32/591 qemu:qtest+qtest-aarch64 / qtest-aarch64/migration-test
> >            ERROR          152.27s   killed by signal 6 SIGABRT
> > ――――――――――――――――――――――――――――――――――――― ✀  
> > ―――――――――――――――――――――――――――――――――――――
> > stderr:
> > query-migrate shows failed migration: Unable to write to socket: Broken pipe
> > **
> > ERROR:../../tests/qtest/migration-helpers.c:151:migrate_query_not_failed:
> > assertion failed: (!g_str_equal(status, "failed"))
> > Assertion failed: (pid == s->qemu_pid), function qtest_wait_qemu, file
> > ../../tests/qtest/libqtest.c, line 207.
> >
> > (test program exited with status code -6)
> > ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
> >
> > and similarly:
> >
> > ▶  34/591 
> > ERROR:../../tests/qtest/migration-helpers.c:151:migrate_query_not_failed:
> > assertion failed: (!g_str_equal(status, "failed")) ERROR
> >  34/591 qemu:qtest+qtest-i386 / qtest-i386/migration-test
> >            ERROR          169.44s   killed by signal 6 SIGABRT
> > ――――――――――――――――――――――――――――――――――――― ✀  
> > ―――――――――――――――――――――――――――――――――――――
> > stderr:
> > query-migrate shows failed migration: Unable to write to socket: Broken pipe
> > **
> > ERROR:../../tests/qtest/migration-helpers.c:151:migrate_query_not_failed:
> > assertion failed: (!g_str_equal(status, "failed"))
> > Assertion failed: (pid == s->qemu_pid), function qtest_wait_qemu, file
> > ../../tests/qtest/libqtest.c, line 207.
> >
> > (test program exited with status code -6)
> > ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
> >
> > It seems to be fairly consistent. Any ideas what it might be?
> > Maybe the QEMU process has already exited before the test binary
> > gets round to querying the status ?
>
> Yes, it sounds like it, can you get a backtrace to figure out which test
> it was in/where it was upto when it died?

The logfile says it had just done
ok 23 /aarch64/migration/multifd/tcp/plain/none
so I think the one it was in the middle of when it failed was
/aarch64/migration/multifd/tcp/plain/cancel.
Similarly the log suggests the x86 failure was for
/i386/migration/miltifd/tcp/plain/cancel.

It doesn't seem to repro running manually, my guess is that
it happens because the machine is heavily loaded doing the
whole build-and-test cycle.

thanks
-- PMM