Re: [Qemu-devel] [PATCH v2 1/1] tests/docker: Add a Avocado Docker test

[Lucas Meneghel Rodrigues]

On Tue, May 22, 2018 at 9:50 PM, Cleber Rosa <address@hidden> wrote:
>
>
> On 05/21/2018 07:32 PM, Philippe Mathieu-Daudé wrote:
>> On 05/21/2018 07:33 PM, Alistair Francis wrote:
>>> On Sun, May 20, 2018 at 8:16 PM, Fam Zheng <address@hidden> wrote:
>>>> On Fri, 05/18 11:34, Alistair Francis wrote:
>>>>> Avocado is not trivial to setup on non-Fedora systems. To simplfying
>>>>> future testing add a docker test image that runs Avocado tests.
>>>>>
>>>>> Signed-off-by: Alistair Francis <address@hidden>
>>>>> ---
>>>>> v2:
>>>>>  - Add a seperate fedora-avocado Docker image
>>>>>  - Move the avocado vt-bootstrap into the Docker file
>>>>>
>>>>>  tests/docker/Makefile.include                 |  1 +
>>>>>  .../docker/dockerfiles/fedora-avocado.docker  | 25 +++++++++++++++++
>>>>>  tests/docker/test-avocado                     | 28 +++++++++++++++++++
>>>>>  3 files changed, 54 insertions(+)
>>>>>  create mode 100644 tests/docker/dockerfiles/fedora-avocado.docker
>>>>>  create mode 100755 tests/docker/test-avocado
>>>>>
>>>>> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
>>>>> index ef1a3e62eb..0e3d108dde 100644
>>>>> --- a/tests/docker/Makefile.include
>>>>> +++ b/tests/docker/Makefile.include
>>>>> @@ -60,6 +60,7 @@ docker-image-debian-ppc64el-cross: docker-image-debian9
>>>>>  docker-image-debian-s390x-cross: docker-image-debian9
>>>>>  docker-image-debian-win32-cross: docker-image-debian8-mxe
>>>>>  docker-image-debian-win64-cross: docker-image-debian8-mxe
>>>>> +docker-image-fedora-avocado: docker-image-fedora
>>>>>  docker-image-travis: NOUSER=1
>>>>>
>>>>>  # Expand all the pre-requistes for each docker image and test combination
>>>>> diff --git a/tests/docker/dockerfiles/fedora-avocado.docker 
>>>>> b/tests/docker/dockerfiles/fedora-avocado.docker
>>>>> new file mode 100644
>>>>> index 0000000000..55b19eebbf
>>>>> --- /dev/null
>>>>> +++ b/tests/docker/dockerfiles/fedora-avocado.docker
>>>>> @@ -0,0 +1,25 @@
>>>>> +FROM qemu:fedora
>>>>> +
>>>>> +ENV PACKAGES \
>>>>> +    libvirt-devel \
>>>>> +    nc \
>>>>> +    python-avocado \
>
> python-avocado is a "Provides" of python2-avocado, but that may may go
> away soon (as soon as Fedora 29).  I'd recommend using python2-avocado,
> which is the proper package name available on Fedora.
>
>>>>> +    python2-devel python3-devel \
>>>>> +    qemu-kvm \
>>>>> +    tcpdump \
>>>>> +    xz
>>>>> +ENV PIP_PACKAGES \
>>>>> +    avocado-qemu \
>
> +CC: Amador Pahim
>
> This pip package was created by Amador Pahim, but was intended for demo
> purposes only.  Upon finding about this, he (cc'ed here) said that the
> package is going to be removed to avoid further confusion.
>
> BTW, the intention is to have this supporting code for QEMU tests in the
> QEMU tree itself.  Myself and Eduardo are working on a new version of
> that supporting code.  Also, based on what I see on the test-avocado
> file, this code is not needed at this point.
>
>>>>> +    avocado-framework-plugin-runner-remote \
>>>>> +    avocado-framework-plugin-runner-vm \
>
> These two plugins are only needed if you intend to run tests on remote
> machines.  Given "test-avocado" is not making use of those features,
> there's no need to install them.
>
> As a side note, plugins are also available on Fedora as RPM packages, so
> if I failed to understand the requirements for these plugins here, they
> could still be installed as the python2-avocado-plugins-runner-remote
> and python2-avocado-plugins-runner-vm RPM packages.
>
>>>>> +    avocado-framework-plugin-vt
>>>>> +
>
> +CC: Lucas Rodrigues
>
> This is a pip package created by Lucas Rodrigues, and this code is
> certainly used in the "test-avocado" file.  He told me he's using it to
> run Avocado-VT tests in a venv on Ubuntu, Amazon Linux, etc, and he's
> willing to maintain and update it.

I'd suggest to look into virtual environments to run avocado-vt.
virtual environments are more lightweight than using a docker
container to run things and achieve similar functionality. Since I
mostly never install python libraries and projects system wide these
days, I didn't test this install mode in a docker container. Here's
what I'd do on my Ubuntu 16.04 development laptop:

$ virtualenv avocado_vt_demo
Running virtualenv with interpreter /usr/bin/python2
New python executable in /home/lmr/avocado_vt_demo/bin/python2
Also creating executable in /home/lmr/avocado_vt_demo/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.

$ source avocado_vt_demo/bin/activate

(avocado_vt_demo) $ pip install avocado-framework-plugin-vt
... output trimmed ...
Successfully built avocado-framework-plugin-vt autotest aexpect simplejson
Installing collected packages: pbr, six, stevedore, avocado-framework,
autotest, aexpect, simplejson, netaddr, netifaces,
avocado-framework-plugin-vt
Successfully installed aexpect-1.4.0 autotest-0.16.4
avocado-framework-61.0 avocado-framework-plugin-vt-57.4 netaddr-0.7.19
netifaces-0.10.7 pbr-4.0.3 simplejson-3.15.0 six-1.11.0
stevedore-1.28.0

(avocado_vt_demo) $ avocado vt-bootstrap --yes-to-all --vt-type qemu
Failed to load plugin from module "avocado_vt.plugins.vt_list":
EnvironmentError("Bootstrap missing. Execute 'avocado vt-bootstrap' or
disable this plugin to get rid of this message",)
Failed to load plugin from module "avocado_vt.plugins.vt":
EnvironmentError("Bootstrap missing. Execute 'avocado vt-bootstrap' or
disable this plugin to get rid of this message",)
Running bootstrap for qemu

1 - Checking the mandatory programs and headers
/usr/bin/xz OK
/usr/sbin/tcpdump OK
/bin/nc OK
/bin/ip OK
/usr/bin/arping OK
/usr/bin/gcc OK
/usr/include/python2.7/Python.h OK
/usr/include/linux/types.h OK
/usr/include/linux/unistd.h OK
/usr/include/linux/socket.h OK

... output trimmed ...

9 - Checking for modules kvm, kvm-intel
Module kvm loaded
Module kvm-intel loaded

VT-BOOTSTRAP FINISHED
You may take a look at the following online docs for more info:
 - http://avocado-vt.readthedocs.org/
 - http://avocado-framework.readthedocs.org/

(avocado_vt_demo) $ avocado run boot
JOB ID     : 2ffba3965cd327e768bd8dadf4f82d6416a4f293
JOB LOG    : /home/lmr/avocado/job-results/job-2018-05-22T22.09-2ffba39/job.log
 (1/1) io-github-autotest-qemu.boot: PASS (31.31 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
| CANCEL 0
JOB TIME   : 33.25 s

I can maintain the pip package mostly in sync with the other avocado
releases. Just keep in mind that my bandwidth for maintenance tasks
outside of my job @Amazon is limited, so you will have to nudge me if
you need something urgently.

Cheers,

Lucas

>>>>> +ENV QEMU_CONFIGURE_OPTS --python=/usr/bin/python3
>
> OK, "python3-avocado" is not yet available on Fedora, because it still
> tracks the 52.x (LTS) series.  So, if Python 3 is the version chosen, it
> maybe a good idea to remove "python-avocado" from "PACKAGES" completely
> and use "avocado-framework" instead on "PIP_PACKAGES".
>
>>>>
>>>> I think this is inherited from qemu:fedora, no?
>>>
>>> It is, I have removed it.
>>>
>>>>
>>>>> +
>>>>> +RUN dnf install -y $PACKAGES
>>>>> +RUN pip install $PIP_PACKAGES
>>>>> +RUN avocado vt-bootstrap --yes-to-all --vt-type qemu
>>>>> +
>>>>> +RUN rpm -q $PACKAGES | sort > /packages.txt
>>>>
>>>> Can you keep the parent image's list with ">>" or appending to the old 
>>>> $PACKAGES
>>>> in the above "ENV" directive?
>>>>
>>>>> +
>>>>> +ENV FEATURES mingw clang pyyaml asan avocado
>>>>
>>>> Similarly, is it possible to append to the parent list instead of 
>>>> overriding?
>>>
>>> I have changed both of these to append to the original variables.
>>
>> Feel free to add my R-b then:
>> Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
>>
>> I'll wait your respin to try again, hoping I got my proxy issues solved.
>>
>>>>
>>>>> diff --git a/tests/docker/test-avocado b/tests/docker/test-avocado
>>>>> new file mode 100755
>>>>> index 0000000000..40474db2ce
>>>>> --- /dev/null
>>>>> +++ b/tests/docker/test-avocado
>>>>> @@ -0,0 +1,28 @@
>>>>> +#!/bin/bash -e
>>>>> +#
>>>>> +# Avocado tests on Fedora, as these are a real pain on Debian systems
>>>>
>>>> Shouldn't pip packages work just well on Debian too? What are the pain?
>>>> (Cc'ing Cleber who may want to know this).
>>>
>>> There is no debian package at the moment.
>>>
>
> That's why Fam said *pip* packages.  In my experience with pip packages
> on Debian:
>
>  * avocado-framework: installation and general usage works well.  Found
> one issue though when running on containers, though, with an easy
> workaround[1]
>
>  * avocado-framework-plugins-vt: currently at version 57.x, and when
> used outside of venvs, doesn't get the "test-providers" directory
> right[2].  Same happened with latest master (using pip install -e
> git+https://github.com/avocado-framework/avocado-vt.git#egg=avocado-plugins-vt)
>
> [1] -
> https://trello.com/c/yEH8ixps/1328-avocado-crashes-on-systems-without-less
>
> [2] -
> https://trello.com/c/0BZZE4CR/1329-avocado-vt-bootstrap-from-pip-package-fails-to-find-test-providers-dir
>
> - Cleber.
>
>>> Alistair
>>>
>>>>
>>>> Fam
>>>>
>>>>> +#
>>>>> +# Copyright (c) 2018 Western Digital.
>>>>> +#
>>>>> +# Authors:
>>>>> +#  Alistair Francis <address@hidden>
>>>>> +#
>>>>> +# This work is licensed under the terms of the GNU GPL, version 2
>>>>> +# or (at your option) any later version. See the COPYING file in
>>>>> +# the top-level directory.
>>>>> +#
>>>>> +# Run this test: NOUSER=1 make address@hidden
>>>>> +
>>>>> +. common.rc
>>>>> +
>>>>> +requires avocado
>>>>> +
>>>>> +cd "$BUILD_DIR"
>>>>> +
>>>>> +DEF_TARGET_LIST="x86_64-softmmu"
>>>>> +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
>>>>> +build_qemu
>>>>> +install_qemu
>>>>> +
>>>>> +export PATH="${PATH}:$(pwd)"
>>>>> +avocado run boot --vt-qemu-bin ./x86_64-softmmu/qemu-system-x86_64
>>>>> --
>>>>> 2.17.0
>>>>>
>>



-- 
Lucas