[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH V5 2/5] runner: Tool for fuzz tests execution
From: |
Fam Zheng |
Subject: |
Re: [Qemu-devel] [PATCH V5 2/5] runner: Tool for fuzz tests execution |
Date: |
Fri, 8 Aug 2014 19:08:17 +0800 |
User-agent: |
Mutt/1.5.23 (2014-03-12) |
On Fri, 08/08 12:58, M.Kustova wrote:
> On Fri, Aug 8, 2014 at 12:50 PM, Fam Zheng <address@hidden> wrote:
> > On Wed, 08/06 17:12, Maria Kustova wrote:
> >> The purpose of the test runner is to prepare the test environment (e.g.
> >> create
> >> a work directory, a test image, etc), execute a program under test with
> >> parameters, indicate a test failure if the program was killed during the
> >> test
> >> execution and collect core dumps, logs and other test artifacts.
> >>
> >> The test runner doesn't depend on an image format or a program will be
> >> tested,
> >> so it can be used with any external image generator and program under test.
> >>
> >> Signed-off-by: Maria Kustova <address@hidden>
> >> ---
> >> tests/image-fuzzer/runner/runner.py | 405
> >> ++++++++++++++++++++++++++++++++++++
> >> 1 file changed, 405 insertions(+)
> >> create mode 100755 tests/image-fuzzer/runner/runner.py
> >>
> >> diff --git a/tests/image-fuzzer/runner/runner.py
> >> b/tests/image-fuzzer/runner/runner.py
> >> new file mode 100755
> >> index 0000000..3fa7fca
> >> --- /dev/null
> >> +++ b/tests/image-fuzzer/runner/runner.py
> >> @@ -0,0 +1,405 @@
> >> +#!/usr/bin/env python
> >> +
> >> +# Tool for running fuzz tests
> >> +#
> >> +# Copyright (C) 2014 Maria Kustova <address@hidden>
> >> +#
> >> +# This program is free software: you can redistribute it and/or modify
> >> +# it under the terms of the GNU General Public License as published by
> >> +# the Free Software Foundation, either version 2 of the License, or
> >> +# (at your option) any later version.
> >> +#
> >> +# This program is distributed in the hope that it will be useful,
> >> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >> +# GNU General Public License for more details.
> >> +#
> >> +# You should have received a copy of the GNU General Public License
> >> +# along with this program. If not, see <http://www.gnu.org/licenses/>.
> >> +#
> >> +
> >> +import sys
> >> +import os
> >> +import signal
> >> +import subprocess
> >> +import random
> >> +import shutil
> >> +from itertools import count
> >> +import getopt
> >> +import StringIO
> >> +import resource
> >> +
> >> +try:
> >> + import json
> >> +except ImportError:
> >> + try:
> >> + import simplejson as json
> >> + except ImportError:
> >> + print >>sys.stderr, \
> >> + "Warning: Module for JSON processing is not found.\n" \
> >> + "'--config' and '--command' options are not supported."
> >> +
> >> +# Backing file sizes in MB
> >> +MAX_BACKING_FILE_SIZE = 10
> >> +MIN_BACKING_FILE_SIZE = 1
> >> +
> >> +
> >> +def multilog(msg, *output):
> >> + """ Write an object to all of specified file descriptors."""
> >> + for fd in output:
> >> + fd.write(msg)
> >> + fd.flush()
> >> +
> >> +
> >> +def str_signal(sig):
> >> + """ Convert a numeric value of a system signal to the string one
> >> + defined by the current operational system.
> >> + """
> >> + for k, v in signal.__dict__.items():
> >> + if v == sig:
> >> + return k
> >> +
> >> +
> >> +def run_app(fd, q_args):
> >> + """Start an application with specified arguments and return its exit
> >> code
> >> + or kill signal depending on the result of execution.
> >> + """
> >> + devnull = open('/dev/null', 'r+')
> >> + process = subprocess.Popen(q_args, stdin=devnull,
> >> + stdout=subprocess.PIPE,
> >> + stderr=subprocess.PIPE)
> >> + out, err = process.communicate()
> >> + fd.write(out)
> >> + fd.write(err)
> >> + return process.returncode
> >> +
> >> +
> >> +class TestException(Exception):
> >> + """Exception for errors risen by TestEnv objects."""
> >> + pass
> >> +
> >> +
> >> +class TestEnv(object):
> >> +
> >> + """Test object.
> >> +
> >> + The class sets up test environment, generates backing and test images
> >> + and executes application under tests with specified arguments and a
> >> test
> >> + image provided.
> >> +
> >> + All logs are collected.
> >> +
> >> + The summary log will contain short descriptions and statuses of tests
> >> in
> >> + a run.
> >> +
> >> + The test log will include application (e.g. 'qemu-img') logs besides
> >> info
> >> + sent to the summary log.
> >> + """
> >> +
> >> + def __init__(self, test_id, seed, work_dir, run_log,
> >> + cleanup=True, log_all=False):
> >> + """Set test environment in a specified work directory.
> >> +
> >> + Path to qemu-img and qemu-io will be retrieved from 'QEMU_IMG' and
> >> + 'QEMU_IO' environment variables.
> >> + """
> >> + if seed is not None:
> >> + self.seed = seed
> >> + else:
> >> + self.seed = str(random.randint(0, sys.maxint))
> >> + random.seed(self.seed)
> >> +
> >> + self.init_path = os.getcwd()
> >> + self.work_dir = work_dir
> >> + self.current_dir = os.path.join(work_dir, 'test-' + test_id)
> >> + self.qemu_img = os.environ.get('QEMU_IMG', 'qemu-img')\
> >> + .strip().split(' ')
> >
> > Nitpicking. I think split(' ') doesn't make sense, this could instead be:
> >
> > self.qemu_img = [os.environ.get('QEMU_IMG', 'qemu-img').strip()]
> >
> > Otherwise user won't be able to pass in a QEMU_IMG path with space.
> >
> > Corner case, though. Otherwise looks good,
>
> This functionality was inherited from qemu-iotests/iotests.py.
> Rationale for this splitting was that environment variables can
> contain call arguments as well.
> Ruining paths with white spaces was a reasonable trade-off.
OK, thanks for explanation.
Fam
- [Qemu-devel] [PATCH V5 0/5] tests: Add the image fuzzer with qcow2 support, Maria Kustova, 2014/08/06
- [Qemu-devel] [PATCH V5 3/5] fuzz: Fuzzing functions for qcow2 images, Maria Kustova, 2014/08/06
- [Qemu-devel] [PATCH V5 4/5] layout: Generator of fuzzed qcow2 images, Maria Kustova, 2014/08/06
- [Qemu-devel] [PATCH V5 5/5] package: Public API for image-fuzzer/runner/runner.py, Maria Kustova, 2014/08/06
- Re: [Qemu-devel] [PATCH V5 0/5] tests: Add the image fuzzer with qcow2 support, Stefan Hajnoczi, 2014/08/08