Re: [Qemu-block] [PATCH 00/17] iotests: Fix iotests for weird formats/options

[Max Reitz]

On 2017-11-23 08:24, Fam Zheng wrote:
> On Thu, 11/23 03:08, Max Reitz wrote:
>> (I hate to write Python tests because the boilerplate seems so large and
>>  the debugging is so hard.  But there is test 194 which shows that it is
>>  possible to write simple bash-like tests as well--and that is how I
>>  should probably write tests from now on.)
> 
> If the boilerplate means copy&paste from another test it is not a big problem;

I do much more C&P in bash tests. ;-)

It's not about the boilerplate I have to write, but the boilerplate that
exists and that tries its best to be opaque to me.

> if you really mean it's more cumbersome than bash to write test code, I think 
> we
> can try new ways to exploit Python's expressiveness. 2c93c5cb43 and f4844ac0ad
> are good examples.

One thing is that I hate to write tests that are methods in classes and
are called automagically by their method name starting in "test_" (or
something?).  I personally don't like the start-up/tear-down semantics
through methods very much either and feel like often it doesn't quite
fit the iotests.

Indeed, I like the blocks Stefan uses in 194 with set-up in the with
statement and implicit tear-down at its end (which f4844ac0ad helps with).

The kind of other thing is that by boilerplate I mean all of this magic.
 We have this QMPTestCase framework behind everything (except 194) and I
have no real idea how it works, and I don't feel like I'm supposed to.

Also, I just personally dislike frameworks in general.  I hate to write
code that's magically invoked by other code without me knowing how, and
that's exactly what's happening with QMPTestCase--you write some classes
and some methods and never are they actually instantiated in your code,
but it's some other code that I don't see (and that I'm supposed to not
care about).  I just don't like that.   I want to write a simple test,
not a library.

Finally, and probably least of the issues, VM.  Which inherits from
QEMUQtestMachine.  Which inherits from QEMUMachine.  All of which are in
different files.  This, too, feels a bit like I'm not supposed to know
what's going on and a bit opaque.  (In contrast to bash's common.qemu,
where I can see exactly what's doing what and don't have to follow
method calls through three files.)  But having to look at three files
instead of one is really a minor issue compared to the rest (but it does
add to the feeling of boilerplate).

So all in all much very personal annoyance.  But still, since I have to
write some tests, this personal annoyance used to result in me
preferring bash for the job.

> So what do you think about debugging? IMO the biggest problem is the less
> verbose reference output, and lack of a separate logging channel. We can and
> should improve that, even for existing scripts.

Yes, those are my issues as well.  Especially that stderr would be my
preferred separate logging channel, but it's redirected and never
printed.  Well, I now have this:

#define loggu(...) \
    do { \
        FILE *fp = fopen("/tmp/log", "a"); \
        fprintf(fp, __VA_ARGS__); \
        fclose(fp); \
    } while (0)

which helps me get around this, but, you know...

> And yes, the 194 style is actually better than "iotests.QMPTestCase" ones.

As you could see, roughly 90 % of my annoyance is due to QMPTestCase, so
I tend to agree. :-)

(The other 10 % is not iotests.VM, but Python itself--and that's
comparing it to bash...)

Max

signature.asc
Description: OpenPGP digital signature