[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] [PATCH 1/2] Add qemu-test for automated testing
|
From: |
Ryan Harper |
|
Subject: |
[Qemu-devel] [PATCH 1/2] Add qemu-test for automated testing |
|
Date: |
Wed, 15 Oct 2008 14:53:33 -0500 |
This patch places the qemu-test framework and tests into the qemu source tree.
There are a number of components to this patch:
- Python-based framework for interacting with the qemu process and the guest
via serial console and qemu monitor.
- Several intial tests cases for exercising various qemu features/functions
- Bash script (run.sh) for running a set of tests on a set of images and
collecting results.
This is a self-contained patch, the framework, tests and script are
self-reliant, no further qemu integration is needed to realize the value of
automatically testing qemu features against a set of images.
Patch 2/2 provides Makefile integration into qemu to execute qemu-test against
the current build of the tree. This can be done without the Makefile
integration, but simplifies the execution.
Signed-off-by: Ryan Harper <address@hidden>
diff --git a/tests/qemu-test/CONFIG b/tests/qemu-test/CONFIG
new file mode 100644
index 0000000..0da0947
--- /dev/null
+++ b/tests/qemu-test/CONFIG
@@ -0,0 +1,58 @@
+###############################################################################
+# QEMU-TEST options and defaults
+#
+###############################################################################
+[options]
+debug=1
+logfile=qemu-test.log
+# Uncomment if you want to redirect guest serial output to a file instead
+# of stdout
+#serialfile=console.log
+
+###############################################################################
+# !!!!WARNING!!!!: don't change the defaults, instead add an override for your
+# distro, see below
+###############################################################################
+[defaults]
+username=root
+password=
+login_prompt=^(\S+) login:
+shell_prompt=(%(username)s@)*%(hostname)s.*#\ $
+
+###############################################################################
+# OVERRIDES
+#
+# Distro overrides section, to override the default values as specified in the
+# [defaults] section, add the same option name and a new value, e.g, overriding
+# the default username:
+#
+# [rhel]
+# username=foobar
+#
+#
+# Fields:
+# username:
+# password:
+# login_prompt:
+# shell_prompt:
+# banner:
+# add a regular expression to match the output from your distro's welcome
banner
+# (/etc/issue) the option name will be used for overriding the default section.
+# For example, if you add the following option:
+#
+###############################################################################
+
+[rhel]
+banner=^Red Hat Enterprise Linux
+shell_prompt=^\[%(username)address@hidden(hostname)s.*\]#\ $
+
+[opensuse]
+banner=^Welcome to openSUSE
+shell_prompt=%(hostname)s:.*#\ $
+
+[ubuntu]
+banner=^Ubuntu
+shell_prompt=^%(username)address@hidden(hostname)s:.*#\ $
+
+[fedora]
+banner=^Fedora
diff --git a/tests/qemu-test/HOWTO b/tests/qemu-test/HOWTO
new file mode 100644
index 0000000..1bac6bc
--- /dev/null
+++ b/tests/qemu-test/HOWTO
@@ -0,0 +1,30 @@
+Configure your Guest
+--------------------
+1. Configure guest to support serial login
+
+
+2. Create an image file in qemu-test/images:
+
+For example:
+
+% cat images/rhel5.2
+DISK="/home/rharper/work/images/rhel5.2_x86_64_10G.qcow2"
+COMMAND="qemu-system-x86_64 -m 512 -smp 2 -net tap -net nic,model=e1000 -net
nic,model=rtl8139 -drive file=${DISK},if=ide,boot=on -vnc none"
+
+You can omit the COMMAND variable and a default qemu command will be run
instead
+(see run.sh). For each guest image or config add additional files in
+qemu-test/images.
+
+3. Run the test and observe the results:
+
+% make && sudo make test
+<snip massive amount of output>
+************************************************************
+Results: 5 passed, 4 FAILED
+passed: e820_memory.py:fedora-9-x86_64
+passed: host_shutdown.py:fedora-9-x86_64
+passed: networking.py:fedora-9-x86_64
+passed: reboot.py:fedora-9-x86_64
+FAILED: Test:host_reset.py,Image:fedora-9-x86_64
+FAILED: Test:timedrift.py,Image:fedora-9-x86_64
+************************************************************
diff --git a/tests/qemu-test/Makefile b/tests/qemu-test/Makefile
new file mode 100644
index 0000000..76c340f
--- /dev/null
+++ b/tests/qemu-test/Makefile
@@ -0,0 +1,24 @@
+-include ../../config-host.mak
+
+TARGET_ARCH := $(shell grep TARGET_ARCH $(QEMU)/config.mak | sed -e
s/TARGET_ARCH=//)
+ifeq ($(TARGET_ARCH), i386)
+QEMU_PROG=qemu$(EXESUF)
+else
+QEMU_PROG=qemu-system-$(TARGET_ARCH)$(EXESUF)
+endif
+
+setup:
+ cd ../../pc-bios && ln -sf ../keymaps
+ chmod a+x ./run.sh
+
+clean:
+ rm -f ../../pc-bios/keymaps
+
+test: all
+
+.PHONY: all test clean setup
+all: setup
+ ./run.sh QEMU="$(QEMU)/$(QEMU_PROG) -L ../../pc-bios" \
+ TESTS="$(TESTS)" ARGS="$(ARGS)" IMAGEDIR="$(IMAGEDIR)" |\
+ tee $(QEMU)_test_results.out
+
diff --git a/tests/qemu-test/THEORY b/tests/qemu-test/THEORY
new file mode 100644
index 0000000..16d5c48
--- /dev/null
+++ b/tests/qemu-test/THEORY
@@ -0,0 +1,4 @@
+Each test should work with as many guests as possible with the least number
+of parameters required.
+
+Minimum guest requirements (only configuration)
diff --git a/tests/qemu-test/TODO b/tests/qemu-test/TODO
new file mode 100644
index 0000000..4a927f8
--- /dev/null
+++ b/tests/qemu-test/TODO
@@ -0,0 +1,33 @@
+* check guest image for commands/utilities needed for each benchmark
+* improve failure detection
+ o oops/soft lockup detection
+ o freeze detection
+ - hard lockup detection (100% cpu usage, min timeout expired)
+ - soft lockup detection (0% cpu usage, min timeout expired)
+ - qemu freeze detection (timeout in monitor command)
+* lots more tests
+ o device validation
+ o disk emu/paravirt verification
+ o pause/resume
+ o savevm/loadvm
+* randomize sockets
+* rewrite networking.py to parse info out of sysfs
+* pass test name to launch for logging
+
+
+#### Completed #####
+2008-08-01
+* write up on how to configure various distros to be ready to run any test
+* redirect serial output to file, print to screen as option
+* detect guest type
+ o automagically select appropriate defaults for prompt
+2008-07-23
+* make check|test integration into kvm-userspace tree
+* fedora guest support
+
+2008-07-16
+* integrate config file
+ o support per-distro overrides
+* ubuntu guest support
+* rhel5 guest support
+* opensuse geust support
diff --git a/tests/qemu-test/boottime.py b/tests/qemu-test/boottime.py
new file mode 100644
index 0000000..f2b2020
--- /dev/null
+++ b/tests/qemu-test/boottime.py
@@ -0,0 +1,38 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <address@hidden
+
+from qemu.test import launch, TimeoutException
+import sys, time
+
+def run_test(vm):
+ start = time.time()
+ vm.wait_for_boot()
+ end = time.time()
+ vm.login()
+ vm.shutdown()
+ vm.wait_for_shutdown()
+ print '\n************'
+ print 'Time to boot: %s seconds'%(end-start)
+ print '************'
+
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm)
+ finally:
+ vm.quit()
+
+ return err
+
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/e820_memory.py b/tests/qemu-test/e820_memory.py
new file mode 100644
index 0000000..028e5bb
--- /dev/null
+++ b/tests/qemu-test/e820_memory.py
@@ -0,0 +1,117 @@
+# Copyright IBM Corp. 2008
+# Authors: Anthony Liguori <address@hidden>
+# : Ryan Harper <address@hidden
+#
+
+from qemu.test import launch, TimeoutException
+import sys
+import ConfigParser, os
+
+
+def check_table_entry(e820, entry, field, value):
+ v1 = e820[entry][field]
+ v2 = value
+ # third field is a string, so strip to trim newlines and such
+ if field == 2:
+ v1 = v1.strip()
+ v2 = v2.strip()
+ if v1 != v2:
+ print 'e820 entry %d field %d should be "%s", found [%s]' %(
+ entry, field, str(value), e820[entry][field])
+ return 1
+ print 'e820[%d][%d] OK' %(entry, field)
+ return 0
+
+
+def compare_table_entry(e820_a, e820_b, entry, field):
+ v1 = e820_a[entry][field]
+ v2 = e820_b[entry][field]
+ # third field is a string, so strip to trim newlines and such
+ if field == 2:
+ v1 = v1.strip()
+ v2 = v2.strip()
+ if v1 != v2:
+ print 'e820 table entry %d field %d do not match' %(entry, field)
+ print '[%s] != [%s]' %(v1, v2)
+ return 1
+
+ print 'e820[%d][%d] match' %(entry, field)
+ return 0
+
+
+def run_test(vm, memory_size):
+ vm.wait_for_boot()
+
+ vm.login()
+
+ e820 = []
+ output = vm.guest_command('dmesg | grep BIOS-e820')
+ for line in output.split('\n'):
+ # handle timestamps in dmesg
+ if line.startswith('['):
+ _, line = line.split('] ', 1)
+
+ words = line.split(' ', 5)
+ start = long('0x%s' % words[2], 0)
+ end = long('0x%s' % words[4], 0)
+ e820.append((start, end, words[5]))
+
+ if len(e820) != 6:
+ print 'Unusual number of e820 entries: %d' % len(e820)
+ return 1
+
+ e820_static = [(0x00000, 0x09fc00, '(usable)'),
+ (0x9fc00, 0x0a0000, '(reserved)'),
+ (0xe8000, 0x100000, '(reserved)')]
+
+ for i in range(3):
+ if compare_table_entry(e820_static, e820, i, 0): return 1
+ if compare_table_entry(e820_static, e820, i, 1): return 1
+ if compare_table_entry(e820_static, e820, i, 2): return 1
+
+ if check_table_entry(e820, 3, 0, 0x100000): return 1
+ if check_table_entry(e820, 3, 2, '(usable)'): return 1
+ if check_table_entry(e820, 4, 2, '(ACPI data)'): return 1
+
+ if (e820[4][1] - e820[4][0]) != (64 << 10):
+ print 'e820 entry 4 has invalid size'
+ return 1
+
+ if e820[4][1] != memory_size:
+ print 'e820 entry 4 has invalid start address'
+ return 1
+
+ # KVM e820 5, 0: 0xffbd000
+ # QEMU e820 5, 0: 0xffc0000
+ if check_table_entry(e820, 5, 0, 0xfffbd000):
+ if check_table_entry(e820, 5, 0, 0xfffc0000): return 1
+ if check_table_entry(e820, 5, 1, 0x100000000): return 1
+ if check_table_entry(e820, 5, 2, '(reserved)'): return 1
+
+ return 0
+
+
+def main(args):
+ memory_size = 128 << 20
+ for i in range(len(args)):
+ if args[i] == '-m':
+ memory_size = long(args[i + 1]) << 20
+ break
+
+ vm = launch(*args)
+ err = 1
+ try:
+ err = run_test(vm, memory_size)
+ vm.shutdown()
+ vm.wait_for_shutdown()
+ finally:
+ vm.quit()
+
+ return err
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/host_reset.py b/tests/qemu-test/host_reset.py
new file mode 100644
index 0000000..6a5e630
--- /dev/null
+++ b/tests/qemu-test/host_reset.py
@@ -0,0 +1,39 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <address@hidden
+#
+
+from qemu.test import launch, TimeoutException
+import sys
+
+def run_test(vm):
+ vm.wait_for_boot()
+ vm.login()
+ vm.guest_command("echo pressing reset button")
+ output = vm.monitor_command('system_reset')
+ vm.wait_for_restart()
+ vm.wait_for_boot()
+ vm.login()
+
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm)
+ vm.shutdown()
+ vm.wait_for_shutdown()
+ finally:
+ vm.quit()
+
+ return err
+
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/host_shutdown.py b/tests/qemu-test/host_shutdown.py
new file mode 100644
index 0000000..9b180c1
--- /dev/null
+++ b/tests/qemu-test/host_shutdown.py
@@ -0,0 +1,34 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <address@hidden
+
+from qemu.test import launch, TimeoutException
+import sys
+
+def run_test(vm):
+ vm.wait_for_boot()
+ vm.login()
+ vm.guest_command("echo pressing power button")
+ output = vm.monitor_command('system_powerdown')
+ if vm.wait_for_shutdown(): return 1
+
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm)
+ finally:
+ vm.quit()
+
+ return err
+
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/iperf.py b/tests/qemu-test/iperf.py
new file mode 100644
index 0000000..0e102c4
--- /dev/null
+++ b/tests/qemu-test/iperf.py
@@ -0,0 +1,75 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <address@hidden
+#
+
+from qemu.test import launch, TimeoutException
+import sys, os.path, popen2
+
+
+def run_test(vm, args):
+ configured_nics = []
+ iperf_url =
"http://internap.dl.sourceforge.net/sourceforge/iperf/iperf-2.0.4.tar.gz";
+ build_iperf = "cd /tmp && wget %s && tar xzf iperf-2.0.4.tar.gz && cd
iperf-2.0.4 && ./configure && make"%(iperf_url)
+ iperf_bin = "/tmp/iperf-2.0.4/src/iperf"
+ iperf_port = "12345"
+
+ vm.wait_for_boot()
+ vm.login()
+
+ for i in range(len(args)):
+ if args[i].startswith('nic') and 'model=' in args[i]:
+ configured_nics.append(args[i].split('model=')[1].split(',')[0])
+
+ if len(configured_nics) < 1:
+ print "this test requires a configured nic"
+ print "try again with: '-net nic,model=<ethernet device>'"
+ return 1
+
+ gw_info = vm.get_gateway_info()
+ dev_info = vm.get_netinfo(gw_info['device'])
+
+ output = vm.guest_command(build_iperf)
+ output = vm.guest_command('%s --help'%(iperf_bin))
+ output = vm.guest_command('%s -D -s -p %s; echo Launching iperf
Daemon'%(iperf_bin, iperf_port))
+
+ vm.log('building iperf on host')
+ (output, input) = popen2.popen2(build_iperf)
+ # dump output on console for user
+ for l in output.readlines():
+ print "host: %s" %(l.strip())
+
+ print "host: running benchmark"
+ results = []
+ for x in range(0,3):
+ cmd = '%s -c %s -p %s' %(iperf_bin, dev_info['address'], iperf_port)
+ vm.log('running host command: %s'%(cmd))
+ (output, input) = popen2.popen2(cmd)
+ o = output.readlines()
+ r = "".join(o)
+ results.append(r)
+ print "host: benchmark complete"
+
+ vm.shutdown()
+ vm.wait_for_shutdown()
+
+ print "".join(results)
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm, args)
+ finally:
+ vm.quit()
+
+ return err
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/migrate.py b/tests/qemu-test/migrate.py
new file mode 100644
index 0000000..270097b
--- /dev/null
+++ b/tests/qemu-test/migrate.py
@@ -0,0 +1,67 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <address@hidden
+# : Anthony Liguori <address@hidden>
+
+from qemu.test import launch, TimeoutException
+import sys, time
+
+def test_alive(vm):
+ rv = vm.guest_command("true; echo $?")
+ if rv != "0":
+ return False
+
+ return True
+
+def main(args):
+ A = launch(*args)
+ A.wait_for_boot()
+ A.login()
+ B = ""
+
+ try:
+ for i in range(10):
+ B = launch(*(args + ['-incoming', 'tcp:localhost:1025']))
+ A.guest_command('echo ------- migration %s -------' %(i))
+
+ B.booted = A.booted
+ B.logged_in = A.logged_in
+ B.hostname = A.hostname
+ B.username = A.username
+ B.password = A.password
+ B.prompt = A.prompt
+
+ if not test_alive(A):
+ print 'Alive test failed before migration'
+ A.quit()
+ B.quit()
+ return 1
+
+ # migrate from A to B
+ A.guest_command('echo starting migration')
+ A.monitor_command('migrate tcp:localhost:1025')
+ A.quit()
+ # test if guest is still alive
+ B.guest_command('echo migration complete')
+ if not test_alive(B):
+ B.log('Alive test failed after migration')
+ B.quit()
+ return 1
+
+ # swap A and B, A is now origin, B migration target
+ A = B
+ B = None
+
+ finally:
+ if A:
+ A.quit()
+ if B:
+ B.quit()
+
+ return 0
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/migrate_networking.py
b/tests/qemu-test/migrate_networking.py
new file mode 100644
index 0000000..64f7da4
--- /dev/null
+++ b/tests/qemu-test/migrate_networking.py
@@ -0,0 +1,73 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <address@hidden
+# : Anthony Liguori <address@hidden>
+
+from qemu.test import launch, TimeoutException
+import sys, time
+
+def test_ping(vm):
+ gw_info = vm.get_gateway_info()
+ if vm.ping(gw_info['address']):
+ return False
+
+ return True
+
+def main(args):
+ A = launch(*args)
+ A.wait_for_boot()
+ A.login()
+ B = ""
+
+ try:
+ for i in range(10):
+ B = launch(*(args + ['-incoming', 'tcp:localhost:1025']))
+ A.guest_command('echo ------- migration %s -------' %(i))
+
+ B.booted = A.booted
+ B.logged_in = A.logged_in
+ B.hostname = A.hostname
+ B.username = A.username
+ B.password = A.password
+ B.prompt = A.prompt
+
+ if not test_ping(A):
+ print 'Ping test failed before migration'
+ A.quit()
+ B.quit()
+ return 1
+
+ # migrate from A to B
+ A.guest_command('echo starting migration')
+ A.monitor_command('migrate tcp:localhost:1025')
+ A.quit()
+ B.guest_command('echo migration complete')
+
+ # after migration, it takes a bit for arp tables to sync
+ B.guest_command('echo waiting for arp to sync')
+ for x in range(0,10):
+ B.guest_command('arp -a && sleep 1')
+
+ # test outgoing network connection
+ if not test_ping(B):
+ B.log('Ping test failed after migration')
+ B.quit()
+ return 1
+
+ # swap A and B, A is now origin, B migration target
+ A = B
+ B = None
+
+ finally:
+ if A:
+ A.quit()
+ if B:
+ B.quit()
+
+ return 0
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/networking.py b/tests/qemu-test/networking.py
new file mode 100644
index 0000000..d0ee2bd
--- /dev/null
+++ b/tests/qemu-test/networking.py
@@ -0,0 +1,80 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <address@hidden
+#
+
+from qemu.test import launch, TimeoutException
+import sys
+
+def run_test(vm, args):
+ configured_nics = []
+ detected_nics = []
+ nic_static = { 'Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139 C+
(rev 20)': 'rtl8319',
+ 'Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev
20)': 'rtl8319',
+ 'Intel Corporation 82540EM Gigabit Ethernet Controller (rev
03)': 'e1000' }
+ drivers_static = { 'e1000': 'e1000', '8139cp': 'rtl8139' }
+ devices = []
+
+ vm.wait_for_boot()
+ vm.login()
+
+ for i in range(len(args)):
+ if args[i].startswith('nic') and 'model=' in args[i]:
+ configured_nics.append(args[i].split('model=')[1].split(',')[0])
+
+ if len(configured_nics) < 1:
+ print "this test requires a configured nic"
+ print "try again with: '-net nic,model=<ethernet device>'"
+ return 1
+
+ output = vm.guest_command('lspci -v | grep Ethernet')
+ for line in output.split('\n'):
+ device = " ".join(line.split()[3:]).replace('\r','')
+ if device in nic_static.keys():
+ detected_nics.append(nic_static[device])
+
+ if len(detected_nics) != len(configured_nics):
+ print 'Failed to detect all configured nics'
+ print 'Configured nics: %s' %(configured_nics)
+ print 'Detected nics: %s' %(detected_nics)
+ return 1
+
+ # build interface to driver mapping array
+ output = vm.guest_command('for d in `ls -1 /sys/class/net/ | grep eth`; do
driver=$(basename `readlink /sys/class/net/$d/device/driver/module`); echo
${driver}:$d; done');
+ for line in output.split('\n'):
+ (driver,device) = line.split(':')
+ device = device.replace('\r','').replace('\n','')
+ devices.append((device,driver))
+
+ gw_info = vm.get_gateway_info()
+
+ # if device is configured and connected to a gateway, do a ping check
+ # FIXME multi-gateway setups not tested
+ for (device,driver) in devices:
+ dev_info = vm.get_netinfo(device)
+ if dev_info['state'] == 'up':
+ if vm.ping(gw_info['address'], iface=device): return 1
+ else:
+ print "device %s, driver %s not enabled in guest, skipping..."
%(device, driver)
+
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm, args)
+ vm.shutdown()
+ vm.wait_for_shutdown()
+ finally:
+ vm.quit()
+
+ return err
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/qemu/__init__.py b/tests/qemu-test/qemu/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qemu-test/qemu/test.py b/tests/qemu-test/qemu/test.py
new file mode 100644
index 0000000..d7d19aa
--- /dev/null
+++ b/tests/qemu-test/qemu/test.py
@@ -0,0 +1,433 @@
+# Copyright IBM Corp. 2008
+# Authors: Anthony Liguori <address@hidden>
+# : Ryan Harper <address@hidden
+#
+
+import os, sys, socket, re, time, select, ConfigParser
+
+class TimeoutException(Exception):
+ def __init__(self, message):
+ Exception.__init__(self)
+ self.message = message
+
+
+class QEMUInstance(object):
+ def __init__(self, pid, serial_fd, monitor_fd, configfile=None):
+ self.configfile = None
+ self.DEBUG = None
+ self.logfile = None
+ self.logfd = None
+ self.pid = pid
+ self.serial = serial_fd
+ self.serialfile = None
+ self.outfd = sys.stdout
+ self.monitor = monitor_fd
+ self.booted = False
+ self.logged_in = False
+ self.distro = None
+ self.login_prompt = None
+ self.shell_prompt = None
+ self.username = None
+ self.hostname = None
+ self.prompt = None
+
+ # parse config file, check if custom path has been provided
+ if configfile:
+ self.configfile = configfile
+ else:
+ # look for CONFIG file in current dir
+ self.configfile = os.path.join(os.getcwd(),'CONFIG')
+
+ # fire up the parser
+ try:
+ self.config = ConfigParser.ConfigParser()
+ self.config.readfp(open(self.configfile))
+ except Exception, e:
+ print 'Failed to parse config: %s'%(self.configfile)
+ print e.message
+ self.quit()
+
+ # fetch common options from config
+ if self.config.has_option("options", "debug"):
+ self.DEBUG = self.config.get("options", "debug")
+ if self.config.has_option("options", "logfile"):
+ self.logfile = self.config.get("options", "logfile")
+ if self.config.has_option("options", "serialfile"):
+ self.serialfile = self.config.get("options", "serialfile")
+
+ # dump header if we're debuggin
+ if self.DEBUG:
+ self.logfd = open(self.logfile, "a")
+
self.log("****************************************************************")
+ self.log("qemu-test v0.1 init")
+
self.log("****************************************************************")
+
+ if self.serialfile:
+ self.outfd = open(self.serialfile, "a")
+ self.log("Writing serial output to %s"%(self.serialfile))
+
+
+
+ # wait for qemu monitor to come up -- ensure it's working
+ self.wait_for_monitor_prompt()
+
+
+ def log(self, message):
+ if self.DEBUG:
+ self.logfd.write('[PID=%s][%s] %s\n' %(self.pid, time.time(),
message))
+ self.logfd.flush()
+
+ ###########################################################
+ # waiting methods
+ ###########################################################
+
+ # base method which reads from the serial line checking to see
+ # the current input matches a designated regular expression
+ # also provide two timers, total (timeout) and activity timeout
+ # (min_activity). The latter is specified when the caller
+ # wants to ensure some output is being generated with some
+ # frequency. Return 'message' in the exception if either
+ # timer fails
+ def wait_for_rexpr(self, rexpr, timeout=None, min_activity=None,
+ message=None):
+ output = ''
+ m = None
+ self.log("wait_for_rexpr(%s) ->"%(rexpr))
+
+ if message == None:
+ message = "rexpr `%s'\n" % rexpr
+
+ if timeout != None:
+ end_time = time.time() + timeout
+ else:
+ end_time = None
+
+ if min_activity == None:
+ min_activity = timeout
+
+ while m == None:
+ now = time.time()
+ if end_time != None and end_time < now:
+ raise TimeoutException(message)
+
+ if end_time != None or min_activity != None:
+ if end_time == None:
+ wait = min_activity
+ else:
+ wait = min(min_activity, end_time - now)
+
+ rdfds, _, __ = select.select([self.serial], [], [], wait)
+ if self.serial not in rdfds:
+ raise TimeoutException(message)
+
+ ch = self.serial.recv(1)
+ if ch == '':
+ break
+ self.outfd.write(ch)
+ self.outfd.flush()
+ output += ch
+ if output.find('\n') != -1:
+ _, line = output.rsplit('\n', 1)
+ #self.log("searching line: [%s] -> [%s]"%(rexpr, line))
+ m = re.search(rexpr, line)
+ self.log("wait_for_rexpr() <-")
+ return output, m
+
+
+ def wait_for_monitor_prompt(self):
+ info = ''
+ self.log("wait_for_monitor_prompt() ->")
+ while not info.endswith('\n(qemu) '):
+ ch = self.monitor.recv(1)
+ if ch == '':
+ break
+ #self.log("got: %s"%(ch))
+ info += ch
+
+ self.log("wait_for_monitor_prompt() <-")
+ return info
+
+
+ def wait_for_linux_password(self, rexpr='^Password: ', timeout=None):
+ self.wait_for_rexpr(rexpr, timeout=timeout)
+
+
+ def wait_for_boot(self, timeout=None):
+ try:
+ self.log("wait_for_boot() ->")
+ self.detect_distro()
+ self.log("after distro detect")
+ if not self.login_prompt:
+ self.login_prompt = self.getconfig("login_prompt")
+ output, m = self.wait_for_rexpr(self.login_prompt, timeout,
min_activity=120,
+ message="Linux login prompt")
+ self.hostname = m.group(1)
+ self.booted = True
+ self.log("wait_for_boot() <-")
+ except TimeoutException, e:
+ print "wait_for_boot error: %s" % e.message
+ self.log("wait_for_boot error: %s" % e.message)
+
+
+ def detect_distro(self, timeout=None):
+ self.log("detect_distro() ->")
+ if self.distro == None:
+ match_map = []
+ self.log("distro not set yet")
+ # list potential distros to check for based on config overrides
+ distros = filter(lambda x: self.config.has_option(x, 'banner'),
+ self.config.sections())
+ self.log("distros in config: %s"%(distros))
+
+ # keep a position mapping
+ for d in distros:
+ match_map.append(d)
+
+ banner = "|".join(map(lambda y: "(%s)" % self.config.get(y,
'banner',
+ raw=True), distros))
+ self.log("distro rexpr: [%s]"%(banner))
+ try:
+ output, m = self.wait_for_rexpr(banner, timeout=None,
min_activity=120,
+ message="Detect distro banner")
+
+ # groups returns a tuple of which parts of the regular
expression
+ # were involved in the match, extracting which group was
+ # not None will give us an index into our regrexpress for each
+ # distro and we can determine which distro we detected
+ matchlist = list(m.groups())
+ for x in range(len(matchlist)):
+ if matchlist[x] != None:
+ self.distro = match_map[x]
+
+ self.log("detected distro: %s"%(self.distro))
+ except TimeoutException, e:
+ self.log("waiting for distro rexpr FAIL: %s"%(e.message))
+ self.distro = 'defaults'
+ # kick the console to spit out a new login prompt
+ # as we probably ate it looking for a banner
+ self.send_to_guest("\n")
+
+
+ self.log("detect_distro() <-")
+
+ def wait_for_shutdown(self, rexpr='(^Power down.|^System halted.)',
timeout=360):
+ output, m = self.wait_for_rexpr(rexpr, timeout, min_activity=120,
+ message="Linux shutdown message")
+ self.booted = False
+
+
+ def wait_for_restart(self, rexpr='.*Restarting system.$', timeout=120):
+ try:
+ output, m = self.wait_for_rexpr(rexpr, timeout,
min_activity=10,
+ message="Linux shutdown
message")
+ except TimeoutException, e:
+ self.log("WARN: didn't detect system restart")
+ pass
+ self.booted = False
+
+
+ def wait_for_quit(self):
+ self.log("wait_for_quit: waitpid() on PID=%d"%(self.pid))
+ os.waitpid(self.pid, 0)
+
+
+ def getconfig(self, field):
+ def __getconfig(self, section, field):
+ self.log("getconfig: fetching section:%s field:%s "%(section,
+ field))
+ try:
+ return self.config.get(section, field, raw=True)
+ except Exception, e:
+ self.log("getconfig: %s"%(e.message))
+ pass
+ return None
+
+ self.log("getconfig() ->")
+ # try looking in distro override section, otherwise defaults
+ value = __getconfig(self, self.distro, field)
+ if value == None:
+ value = __getconfig(self, "defaults", field)
+
+ self.log("getconfig() <-")
+ return value
+
+
+ def login(self):
+ self.log("login() ->")
+ if not self.prompt:
+ username = self.getconfig("username")
+ password = self.getconfig("password")
+ prompt = self.getconfig("shell_prompt")
+
+ if username != 'root':
+ raise Exception("login requires username=root");
+
+ if not self.booted:
+ self.wait_for_boot()
+
+ self.username = username
+ self.password = password
+ self.prompt = prompt % {'username': username, 'hostname':
self.hostname}
+
+
+ # XXX: maybe I should check for self.booted here as well
+ self.send_to_guest('%s\n' % self.username)
+ self.wait_for_rexpr('^Password: ')
+ # There seems to be a race with the password entry
+ time.sleep(1)
+ self.send_to_guest('%s\n' % self.password)
+ self.wait_for_rexpr(self.prompt, timeout=10)
+ self.logged_in = True
+
+
+ def send_to_guest(self, data):
+ self.serial.sendall(data)
+
+
+ def __sanitize(self, string):
+ return string.replace('\r','')
+
+
+ # common networking helpers
+ def get_gateway_info(self):
+ cmd = "route -n | grep ^0 | awk '{print $8\":\"$2}'"
+ output = self.__sanitize(self.guest_command(cmd))
+ # FIXME: there may be multiple gateways, we're only going to look at
the first
+ # one
+ output = output.split('\n')[0].split(':')
+ return { 'device':output[0], 'address': output[1] }
+
+
+ def get_netinfo(self, devicename):
+ d = self.__sanitize(devicename)
+ netinfo = {}
+ c = "ifconfig %s | grep [A-Za-z]" %(d)
+ ifconfig_raw = self.guest_command(c).split('\n')
+ if 'error' in ifconfig_raw:
+ return netinfo
+
+ netinfo['device'] = d
+ netinfo['macaddr'] = ifconfig_raw[0].split()[-1:]
+ #FIXME: counting # of lines of ifconfig output isn't a good way
+ # to determine if the device is configured
+ if len(ifconfig_raw) >= 8:
+ netinfo['address'] = ifconfig_raw[1].split()[1].split(':')[1]
+ netinfo['netmask'] = ifconfig_raw[1].split()[-1:][0].split(':')[1]
+ if 'UP' in ifconfig_raw[3]:
+ netinfo['state'] = 'up'
+ else:
+ netinfo['state'] = 'down'
+ netinfo['mtu'] = ifconfig_raw[3].split()[4].split(':')[1]
+ else:
+ # device not configured
+ netinfo['state'] = 'down'
+
+ return netinfo
+
+
+ def ping(self, address, iface=None):
+ address = self.__sanitize(address)
+ cmd = "ping -c 5 %s -w 10" %(address)
+ if iface:
+ iface = self.__sanitize(iface)
+ cmd = "%s -I %s" %(cmd, iface)
+ cmd = "%s; echo $?" %(cmd)
+ raw = self.guest_command(cmd).split('\n')
+ # extract the return code, dropping any empty entries from the list
+ rc = filter(lambda x: len(x) > 0, raw)[-1]
+ return int(rc)
+
+
+ def shutdown(self):
+ self.send_to_guest('halt\n')
+
+
+ def quit(self):
+ # might be racy with guest shutting down, so catch broken pipe
+ # exceptions
+ try:
+ self.monitor_command("quit")
+ except Exception, e:
+ self.log("exception: %s"%(e.message))
+ pass
+ finally:
+ self.wait_for_quit()
+
+ def guest_command(self, command, timeout=300):
+ self.log("guest_command(command=%s) ->"%(command))
+ self.send_to_guest('%s\n' % command)
+ output, _ = self.wait_for_rexpr('%s' % self.prompt, timeout,
+ message="Guest command: %s"%(command))
+ try:
+ output, _ = output.rsplit('\n', 1)
+ _, output = output.split('\n', 1)
+ except Exception, e:
+ pass
+ self.log("guest_command() <-")
+ return output.rsplit('\r', 1)[0]
+
+
+ def monitor_command(self, command):
+ self.log("monitor_command(command=%s) ->"%(command))
+ self.monitor.sendall('%s\n' % command)
+ info = self.wait_for_monitor_prompt()
+ self.log("info = %s"%(info))
+ index = info.find('\n')
+ if index != -1:
+ info = info[index+1:]
+ index = info.rfind('\n')
+ if index != -1:
+ info = info[0:index]
+ self.log("monitor_command() <-")
+ return info
+
+
+def launch(executable, *args):
+ def __extract_config(args):
+ if 'config=' in args:
+ configfile = args.split("config=")[1].split()[0]
+ newargs = " ".join(args.split("config=")[1].split()[1:])
+ return (configfile, newargs)
+
+ return (None, args)
+
+
+ (config, args) = __extract_config(args)
+
+ serial_path = os.tmpnam()
+ monitor_path = os.tmpnam()
+
+ try:
+ os.unlink(serial_path)
+ except Exception, e:
+ pass
+ try:
+ os.unlink(monitor_path)
+ except Exception, e:
+ pass
+
+ serial = socket.socket(socket.AF_UNIX)
+ serial.bind(serial_path)
+ serial.listen(1)
+
+ monitor = socket.socket(socket.AF_UNIX)
+ monitor.bind(monitor_path)
+ monitor.listen(1)
+
+ pid = os.fork()
+ if pid == 0:
+ os.execvp(executable, (executable, '-serial', 'unix:%s' % serial_path,
+ '-monitor', 'unix:%s' % monitor_path) + args)
+ sys.exit(1)
+
+ serial_fd, _ = serial.accept()
+ monitor_fd, _ = monitor.accept()
+
+ os.unlink(serial_path)
+ os.unlink(monitor_path)
+
+ serial.close()
+ monitor.close()
+
+ return QEMUInstance(pid, serial_fd, monitor_fd, config)
+
diff --git a/tests/qemu-test/reboot.py b/tests/qemu-test/reboot.py
new file mode 100644
index 0000000..6585c01
--- /dev/null
+++ b/tests/qemu-test/reboot.py
@@ -0,0 +1,42 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <address@hidden
+#
+
+from qemu.test import launch, TimeoutException
+import sys
+
+def run_test(vm):
+ reboots=10
+ vm.wait_for_boot()
+ vm.login()
+
+ for x in range(1,reboots+1):
+ # background the reboot command so we match the prompt string
+ vm.guest_command("echo Reboot %s; reboot&"%(x))
+ vm.wait_for_restart()
+ vm.wait_for_boot()
+ vm.login()
+
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm)
+ vm.shutdown()
+ vm.wait_for_shutdown()
+ finally:
+ vm.quit()
+
+ return err
+
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/run.sh b/tests/qemu-test/run.sh
new file mode 100755
index 0000000..4064c63
--- /dev/null
+++ b/tests/qemu-test/run.sh
@@ -0,0 +1,69 @@
+#!/bin/bash
+#
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <address@hidden
+#
+
+while [ $# -gt 0 ]; do
+ var=${1%=*}
+ value=${1#*=}
+ export ${var}="${value}"
+ shift;
+done
+
+[ -z "${QEMU}" ] && { QEMU="qemu-system-x86_64"; }
+[ -z "${TESTS}" ] && { TESTS="`ls *.py`"; }
+[ -z "${ARGS}" ] && { ARGS="-m 128 -net tap -net nic,model=e1000 -net
nic,model=rtl8139 -snapshot -vnc none"; }
+[ -z "${IMAGEDIR}" ] && { IMAGEDIR=$PWD/images; }
+
+# source repositories don't like empty files very well
+# just in case, touch the file for qemu to ensure the module
+# gets compiled
+touch qemu/__init__.py
+
+mkdir -p ${IMAGEDIR}
+DISKS=( `ls ${IMAGEDIR}` )
+# error out if we don't have any images defined
+[ address@hidden -le 0 ] && {
+ echo "No test images defined in ${IMAGEDIR}"
+ exit 0;
+}
+
+declare -a PASS
+declare -a FAIL
+for t in address@hidden; do
+ for d in address@hidden; do
+ . ${IMAGEDIR}/${d}
+ echo "Running Test:${t} on Image:${d}"
+ if [ -z "${COMMAND}" ]; then
+ COMMAND="${QEMU} ${ARGS} -drive file=${DISK},if=ide
-boot c"
+ echo "Using default command: ${COMMAND}"
+ else
+ # filter out QEMU from COMMAND, image files should use
+ # QEMU if they need to specify, and force -vnc none
+ COMMAND="${QEMU} $(echo $COMMAND | fmt -w 1 | grep -v
${QEMU} | fmt -w 255) -vnc none"
+ echo "Using override command: ${COMMAND}"
+ fi
+ sleep 2
+ sudo python ${t} ${COMMAND}
+ rv="$?"
+ if [ "${rv}" == "0" ]; then
+ address@hidden"${t}:${d}"
+ else
+ address@hidden"Test:${t},Image:${d}"
+ fi
+ # reset command for next image
+ COMMAND=""
+ done
+done
+
+echo ""
+echo "************************************************************"
+echo "Results: address@hidden passed, address@hidden FAILED"
+for f in address@hidden; do
+ echo "passed: ${f}"
+done
+for f in address@hidden; do
+ echo "FAILED: ${f}"
+done
+echo "************************************************************"
diff --git a/tests/qemu-test/timedrift.py b/tests/qemu-test/timedrift.py
new file mode 100644
index 0000000..4c8ab04
--- /dev/null
+++ b/tests/qemu-test/timedrift.py
@@ -0,0 +1,67 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <address@hidden
+#
+
+from qemu.test import launch, TimeoutException
+import sys, time
+
+def run_test(vm):
+ badclocks = []
+ vm.wait_for_boot()
+ vm.login()
+
+ # calculate how much time it takes to run a very simple guest command
+ length = 600
+ now = time.time()
+ vm.guest_command("echo true")
+ end = time.time()
+ overhead = int(end - now)
+ # obviously it takes *some* amount of time to issue the command
+ if overhead == 0:
+ overhead = 1
+ vm.guest_command("echo calculated overhead: %s seconds"%(overhead))
+
+ available = vm.guest_command("cat
/sys/devices/system/clocksource/clocksource0/available_clocksource")
+ clocksource = vm.guest_command("cat
/sys/devices/system/clocksource/clocksource0/current_clocksource")
+ for clock in available.split():
+ if clock == "jiffies":
+ vm.guest_command("echo skipping crappy clock=jiffies")
+ continue
+ vm.guest_command("echo switching clocksource to %s"%(clock))
+ vm.guest_command("echo %s | dd
of=/sys/devices/system/clocksource/clocksource0/current_clocksource" %(clock))
+ now = time.time()
+ vm.guest_command("date -u +%s; sleep 600; date -u +%s", timeout=1200)
+ end = time.time()
+ delta = int(end - now)
+ vm.guest_command("echo %s took %s seconds to complete"%(clock, delta))
+
+ if (delta-length) > overhead:
+ badclocks.append(clock)
+
+ if len(badclocks) > 0:
+ print "Bad clocks: %s"%(badclocks)
+ return 1
+
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm)
+ vm.shutdown()
+ vm.wait_for_shutdown()
+ finally:
+ vm.quit()
+
+ return err
+
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)