Re: [PULL 00/14] Docs / testing patches for 2024-07-29

[Peter Maydell]

On Tue, 30 Jul 2024 at 11:05, Philippe Mathieu-Daudé <philmd@linaro.org> wrote:
>
> On 29/7/24 17:36, Daniel P. Berrangé wrote:
> > On Mon, Jul 29, 2024 at 05:27:00PM +0200, Philippe Mathieu-Daudé wrote:
>
> >> Thomas Huth (9):
> >>    python: Install pycotap in our venv if necessary
> >>    tests/functional: Add base classes for the upcoming pytest-based tests
> >>    tests/Makefile.include: Increase the level of indentation in the help
> >>      text
> >>    tests/functional: Prepare the meson build system for the functional
> >>      tests
> >>    tests/functional: Convert simple avocado tests into standalone python
> >>      tests
> >>    tests/functional: Convert avocado tests that just need a small
> >>      adjustment
> >>    tests/functional: Convert the x86_cpu_model_versions test
> >>    tests/functional: Convert the riscv_opensbi avocado test into a
> >>      standalone test
> >>    gitlab-ci: Add "check-functional" to the build tests
> >
> > On the avocado review I mentioned that I didn't think we should
> > be rebasing avocado during freeze. By the same token, I'm not
> > convinced we should be introducing a new test framework during
> > the freeze period.  This is alot simpler than avocado, but at
> > the same time this small subset of Thomas' patches isn't really
> > fixing any problem on its own, as avocado still exists in
> > parallel.
>
> - As you noticed, Avocado isn't rebased here
> - The python file added is almost a copy of the avocado base class,
>    already well tested
> - Test are moved from Avocado, so no waste of time running them
>    twice
> - macOS developers can run 178 functional tests. This is the big
>    win and time saver for me.
>
> Do you really see merging these few patches now as being
> counter productive for the community?

I do feel from the discussion in other threads that "what is
the way forward for our currently-in-avocado test cases?" is
currently still an open question, so I tend to agree with Daniel
that it would be better to resolve that question first,
and we don't need to be in a hurry to change things in
our test framework during the freeze period.

thanks
-- PMM