[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PULL 19/30] scripts/simplebench: use standard deviation for +- error
From: |
Max Reitz |
Subject: |
[PULL 19/30] scripts/simplebench: use standard deviation for +- error |
Date: |
Fri, 18 Dec 2020 16:12:38 +0100 |
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Standard deviation is more usual to see after +- than current maximum
of deviations.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-16-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
scripts/simplebench/simplebench.py | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/scripts/simplebench/simplebench.py
b/scripts/simplebench/simplebench.py
index 2251cd34ea..55ec1ad5db 100644
--- a/scripts/simplebench/simplebench.py
+++ b/scripts/simplebench/simplebench.py
@@ -18,6 +18,8 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
+import statistics
+
def bench_one(test_func, test_env, test_case, count=5, initial_run=True):
"""Benchmark one test-case
@@ -40,7 +42,7 @@ def bench_one(test_func, test_env, test_case, count=5,
initial_run=True):
'dimension': dimension of results, may be 'seconds' or 'iops'
'average': average value (iops or seconds) per run (exists only if at
least one run succeeded)
- 'delta': maximum delta between test_func result and the average
+ 'stdev': standard deviation of results
(exists only if at least one run succeeded)
'n-failed': number of failed runs (exists only if at least one run
failed)
@@ -67,10 +69,9 @@ def bench_one(test_func, test_env, test_case, count=5,
initial_run=True):
assert all('seconds' in r for r in succeeded)
assert all('iops' not in r for r in succeeded)
dim = 'seconds'
- avg = sum(r[dim] for r in succeeded) / len(succeeded)
result['dimension'] = dim
- result['average'] = avg
- result['delta'] = max(abs(r[dim] - avg) for r in succeeded)
+ result['average'] = statistics.mean(r[dim] for r in succeeded)
+ result['stdev'] = statistics.stdev(r[dim] for r in succeeded)
if len(succeeded) < count:
result['n-failed'] = count - len(succeeded)
@@ -81,7 +82,7 @@ def bench_one(test_func, test_env, test_case, count=5,
initial_run=True):
def ascii_one(result):
"""Return ASCII representation of bench_one() returned dict."""
if 'average' in result:
- s = '{:.2f} +- {:.2f}'.format(result['average'], result['delta'])
+ s = '{:.2f} +- {:.2f}'.format(result['average'], result['stdev'])
if 'n-failed' in result:
s += '\n({} failed)'.format(result['n-failed'])
return s
--
2.29.2
- [PULL 08/30] block/io: bdrv_wait_serialising_requests_locked: drop extra bs arg, (continued)
- [PULL 08/30] block/io: bdrv_wait_serialising_requests_locked: drop extra bs arg, Max Reitz, 2020/12/18
- [PULL 09/30] block: bdrv_mark_request_serialising: split non-waiting function, Max Reitz, 2020/12/18
- [PULL 11/30] block: bdrv_check_perm(): process children anyway, Max Reitz, 2020/12/18
- [PULL 05/30] block: simplify comment to BDRV_REQ_SERIALISING, Max Reitz, 2020/12/18
- [PULL 10/30] block: introduce BDRV_REQ_NO_WAIT flag, Max Reitz, 2020/12/18
- [PULL 14/30] iotests: qemu_io_silent: support --image-opts, Max Reitz, 2020/12/18
- [PULL 13/30] qemu-io: add preallocate mode parameter for truncate command, Max Reitz, 2020/12/18
- [PULL 12/30] block: introduce preallocate filter, Max Reitz, 2020/12/18
- [PULL 15/30] iotests.py: execute_setup_common(): add required_fmts argument, Max Reitz, 2020/12/18
- [PULL 16/30] iotests: add 298 to test new preallocate filter driver, Max Reitz, 2020/12/18
- [PULL 19/30] scripts/simplebench: use standard deviation for +- error,
Max Reitz <=
- [PULL 17/30] scripts/simplebench: fix grammar: s/successed/succeeded/, Max Reitz, 2020/12/18
- [PULL 18/30] scripts/simplebench: support iops, Max Reitz, 2020/12/18
- [PULL 20/30] simplebench: rename ascii() to results_to_text(), Max Reitz, 2020/12/18
- [PULL 23/30] simplebench/results_to_text: add difference line to the table, Max Reitz, 2020/12/18
- [PULL 22/30] simplebench/results_to_text: improve view of the table, Max Reitz, 2020/12/18
- [PULL 21/30] simplebench: move results_to_text() into separate file, Max Reitz, 2020/12/18
- [PULL 24/30] simplebench/results_to_text: make executable, Max Reitz, 2020/12/18
- [PULL 25/30] scripts/simplebench: add bench_prealloc.py, Max Reitz, 2020/12/18
- [PULL 26/30] quorum: Implement bdrv_co_block_status(), Max Reitz, 2020/12/18
- [PULL 27/30] quorum: Implement bdrv_co_pwrite_zeroes(), Max Reitz, 2020/12/18