Re: [PATCH v2] tests/functional/aarch64: add tests for FEAT_RME

[Pierrick Bouvier]

On 12/2/24 02:57, Peter Maydell wrote:
> On Sun, 1 Dec 2024 at 18:09, Pierrick Bouvier
> <pierrick.bouvier@linaro.org> wrote:
> > Hi Marcin,
> >
> > On 12/1/24 05:34, Marcin Juszkiewicz wrote:
> > > W dniu 28.11.2024 o 22:37, Pierrick Bouvier pisze:
> > > > This boot an OP-TEE environment, and launch a nested guest VM inside it
> > > > using the Realms feature. We do it for virt and sbsa-ref platforms.
> > > >
> > > > Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
> > >
> > > > diff --git a/tests/functional/meson.build b/tests/functional/meson.build
> > > > index 5c048cfac6d..b975a1560df 100644
> > > > --- a/tests/functional/meson.build
> > > > +++ b/tests/functional/meson.build
> > > > @@ -13,6 +13,8 @@ endif
> > > >     test_timeouts = {
> > > >       'aarch64_aspeed' : 600,
> > > >       'aarch64_raspi4' : 480,
> > >
> > > > +  'aarch64_rme_virt' : 720,
> > >
> > > Took 2974.95s on M1 Pro macbook.
> > >
> > > > +  'aarch64_rme_sbsaref' : 720,
> > >
> > > This one needed 2288.29s.
> > >
> > > >       'aarch64_sbsaref_alpine' : 720,
> > >
> > > Have to check cause timed out.
> > >
> > > >       'aarch64_sbsaref_freebsd' : 720,
> > >
> > > 331.65s
> > >
> > > So RME tests probably need longer timeouts or would not run at all.
> >
> > By any chance, are you running those tests in debug mode?
> > It seems to me that CI is running functional tests with optimized
> > builds, so I'm not sure we want to support debug "times" here.
>
> We do need to support debug times, because a common developer
> use case is "doing a debug build, run 'make check-functional'
> to check whether anything is broken. The debug times also
> are useful because the CI runners can have highly variable
> performance -- if a test is slow enough to hit the timeout
> for a debug build locally, it's probably going to also hit
> the timeout at least sometimes in CI.

I understand the scenario, but given how slow debug builds are, it would 
probably be faster to advise developer to recompile in release mode.
The overall time of compile + test is slower than waiting for debug.

Beyond using a debugger, what is the advantage to compile with -O0?

> thanks
> -- PMM