[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v7 09/11] iotests: add testrunner.py
From: |
Kevin Wolf |
Subject: |
Re: [PATCH v7 09/11] iotests: add testrunner.py |
Date: |
Fri, 22 Jan 2021 15:51:26 +0100 |
Am 16.01.2021 um 14:44 hat Vladimir Sementsov-Ogievskiy geschrieben:
> Add TestRunner class, which will run tests in a new python iotests
> running framework.
>
> There are some differences with current ./check behavior, most
> significant are:
> - Consider all tests self-executable, just run them, don't run python
> by hand.
> - Elapsed time is cached in json file
> - Elapsed time precision increased a bit
> - use python difflib instead of "diff -w", to ignore spaces at line
> ends strip lines by hand. Do not ignore other spaces.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Sorry for sending so many individual mails, but I'm running into things
as I actually run the code during the review of the final 'check'
rewrite.
I get this output for a failing test case:
001 fail [15:44:41] [15:44:42] 0.5s (last: 13.2s) output
mismatch (see {f_bad})
--- /home/kwolf/source/qemu/tests/qemu-iotests/001.out
+++ 001.out.bad
@@ -1,4 +1,4 @@
-QA output created by 001
+Broken QA output created by 001
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
== reading whole image ==
Failures: 001
Somehow we get additional empty lines in the diff header that shouldn't
be there. Not sure where it comes from. Especially the empty line after
@@ even corrupts the patch if you try to apply it.
The other problem is obvious enough:
> + diff = file_diff(str(f_reference), str(f_bad))
> + if diff:
> + return TestResult(status='fail', elapsed=elapsed,
> + description='output mismatch (see {f_bad})',
This needs to be an f-string.
Kevin
- Re: [PATCH v7 07/11] iotests: add findtests.py, (continued)
Re: [PATCH v7 07/11] iotests: add findtests.py, Kevin Wolf, 2021/01/22
[PATCH v7 09/11] iotests: add testrunner.py, Vladimir Sementsov-Ogievskiy, 2021/01/16
[PATCH v7 10/11] iotests: rewrite check into python, Vladimir Sementsov-Ogievskiy, 2021/01/16
[PATCH v7 06/11] iotests: define group in each iotest, Vladimir Sementsov-Ogievskiy, 2021/01/16
Re: [PATCH v7 00/11] Rework iotests/check, Eric Blake, 2021/01/20