Re: [PULL 00/14] Docs / testing patches for 2024-07-29

[Philippe Mathieu-Daudé]

On 30/7/24 12:14, Peter Maydell wrote:
> On Tue, 30 Jul 2024 at 11:05, Philippe Mathieu-Daudé <philmd@linaro.org> wrote:
> > On 29/7/24 17:36, Daniel P. Berrangé wrote:
> > > On Mon, Jul 29, 2024 at 05:27:00PM +0200, Philippe Mathieu-Daudé wrote:
> >
> > > > Thomas Huth (9):
> > > >     python: Install pycotap in our venv if necessary
> > > >     tests/functional: Add base classes for the upcoming pytest-based tests
> > > >     tests/Makefile.include: Increase the level of indentation in the help
> > > >       text
> > > >     tests/functional: Prepare the meson build system for the functional
> > > >       tests
> > > >     tests/functional: Convert simple avocado tests into standalone python
> > > >       tests
> > > >     tests/functional: Convert avocado tests that just need a small
> > > >       adjustment
> > > >     tests/functional: Convert the x86_cpu_model_versions test
> > > >     tests/functional: Convert the riscv_opensbi avocado test into a
> > > >       standalone test
> > > >     gitlab-ci: Add "check-functional" to the build tests
> > >
> > > On the avocado review I mentioned that I didn't think we should
> > > be rebasing avocado during freeze. By the same token, I'm not
> > > convinced we should be introducing a new test framework during
> > > the freeze period.  This is alot simpler than avocado, but at
> > > the same time this small subset of Thomas' patches isn't really
> > > fixing any problem on its own, as avocado still exists in
> > > parallel.
> >
> > - As you noticed, Avocado isn't rebased here
> > - The python file added is almost a copy of the avocado base class,
> >     already well tested
> > - Test are moved from Avocado, so no waste of time running them
> >     twice
> > - macOS developers can run 178 functional tests. This is the big
> >     win and time saver for me.
> >
> > Do you really see merging these few patches now as being
> > counter productive for the community?
>
> I do feel from the discussion in other threads that "what is
> the way forward for our currently-in-avocado test cases?" is
> currently still an open question,

Thomas demonstrated with few LoC we can leverage some Avocado
libraries (util, archive, ...) without having to use Avocado
framework as our entry point to run acceptance tests - which
is what gave us headaches the last 2 years -.

Richard only requirement is a "fetch assets" equivalent. We'll
look at.

I don't see anything else blocking this test conversion.

> so I tend to agree with Daniel
> that it would be better to resolve that question first,
> and we don't need to be in a hurry to change things in
> our test framework during the freeze period.

OK. I'll repost without these patches.

Regards,

Phil.