Re: [PATCH v3 00/11] tests/migration-test: Allow testing older machine types

[Daniel P . Berrangé]

On Wed, Oct 18, 2023 at 11:19:01AM -0300, Fabiano Rosas wrote:
> Fabiano Rosas <farosas@suse.de> writes:
> 
> > This adds support for running migration-test with two different QEMU
> > versions to test migration compatibility. The tests automatically
> > choose the latest machine type supported by both QEMU versions.
> 
> Let me point out two situations that came up while developing this:
> 
> 1) There was a generic command line change in QEMU (-audio option)
>    introduced this cycle that was ported to the command line used by
>    libqtest.c. This means that when using the same command line to
>    start both guests, the older one will not recognize the option and
>    will break.
> 
>    Ideally, we would keep command lines out of the library and require
>    each test to provide their own. Then maybe the migration code could
>    cope with the change somehow.

Right the problem is that QEMU doesn't provide a long term stable
API, but has finite sliding windows where QEMU versions can be
compatible in their CLI, determined by our minimal deprecation
period of 2 releases.

eg you can come up with a CLI that is compatible for all of 7.1, 7.2
and 8.0 but you can't neccessarily come up with a CLI that is compatible
for all of 7.1, 7.2, 8.0 and 8.1 because the lifespan of that set of
versions is greater than our deprecation period.

If we did't use -audio in libqtest, we could be compatible with current
QEMU and some old QEMUs, but not future QEMU.

If we do use -audio in libqtest, we could be compatible with current
QEMU and some future QEMUs, but not old QEMUs.

This is a no win scenario.

Now, libvirt provides a long term stable configuration for QEMU that
transcends changes in QEMU CLI.

So we could come up with a live migration test between QEMU's of an
arbitrary age difference, if we used libvirt to drive the process.
That wouldn't really be qtest though. Could perhaps we done in the
context of avocado, if we used the libvirt-python API binding, but
this would be different from other avocado tests by virtue of not
using the QEMU python classes for launch.

> 2) When fixing an issue during a release cycle, that issue can still
>    show up on the next one when using this series to test
>    compatibility. There's probably not much we can do here, but
>    remember the issues that were already fixed and ignore them when
>    they appear in these compatibiliy tests.
> 
> I don't see how to avoid the situations above, but I think their
> existence implies we cannot enable by default any sort of migration
> compatibility testing that uses a live guest.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|