Re: [PATCH 0/9] tests, python: prepare to expand usage of test venv

[Paolo Bonzini]

On 5/26/22 16:34, John Snow wrote:
> (1) If check is engaged without running check-venv first and iotests 
> creates its own venv, the python binary it uses will be whichever one is 
> your system default, not necessarily the one you configured your build with.
>
> This is reasonable behavior IMO, but if you later run "make check", 
> there's no guarantee that Make will re-make the venv with the correct 
> python binary  That's a subtle landmine.

Yup, that's also a reason to make initial venv creation part of 
configure.  I consider that on the same level as running Meson and 
setting up git submodules.

> (2) Similarly, if the venv requirements.txt (or python/setup.cfg) 
> change, iotests doesn't have the logic to notice it ought to re-make the 
> venv. Only the makefile does. However, at least in this case, the 
> makefile is guaranteed to notice if/when we run "check block" again. The 
> odds of these files changing for most people who aren't *me* are pretty 
> low, so it may not really come up much. Still, it's not bullet-proof.

Yeah, this is fine.  Compare it with e.g. running clang-query: it needs 
a "make" first to rebuild compile_commands.json.

Paolo

> (Bonus not-at-all-subtle problem) I need to remove iotest 297, otherwise 
> iotests under a venv that doesn't install mypy/pylint will never run. I 
> discussed this upstream recently w/ Kevin, but my series to address it 
> isn't ready yet. (Just pre-emptively pointing it out to say I'm aware of 
> it!)