Re: [Qemu-devel] [PATCH v2 1/1] tests/docker: Add a Avocado Docker test

[Lucas Meneghel Rodrigues]

Me and Cleber were talking about running avocado in containers, and we
verified there are some things we need to fix in avocado to make it
work well under container environments. One of the main offenders is
our service manager library, that was created to deal with sysvinit
and systemd, but Docker containers usually don't have either. The test
runner does use the service library to infer a few things about the
system (even on simple tests), so fixing is needed.

So your docker proposal as of is now depends on us fixing those bugs
for it to be functional. Please bear with us meanwhile.

On Tue, May 22, 2018 at 10:14 PM, Lucas Meneghel Rodrigues
<address@hidden> wrote:
> On Tue, May 22, 2018 at 9:50 PM, Cleber Rosa <address@hidden> wrote:
>>
>>
>> On 05/21/2018 07:32 PM, Philippe Mathieu-Daudé wrote:
>>> On 05/21/2018 07:33 PM, Alistair Francis wrote:
>>>> On Sun, May 20, 2018 at 8:16 PM, Fam Zheng <address@hidden> wrote:
>>>>> On Fri, 05/18 11:34, Alistair Francis wrote:
>>>>>> Avocado is not trivial to setup on non-Fedora systems. To simplfying
>>>>>> future testing add a docker test image that runs Avocado tests.
>>>>>>
>>>>>> Signed-off-by: Alistair Francis <address@hidden>
>>>>>> ---
>>>>>> v2:
>>>>>>  - Add a seperate fedora-avocado Docker image
>>>>>>  - Move the avocado vt-bootstrap into the Docker file
>>>>>>
>>>>>>  tests/docker/Makefile.include                 |  1 +
>>>>>>  .../docker/dockerfiles/fedora-avocado.docker  | 25 +++++++++++++++++
>>>>>>  tests/docker/test-avocado                     | 28 +++++++++++++++++++
>>>>>>  3 files changed, 54 insertions(+)
>>>>>>  create mode 100644 tests/docker/dockerfiles/fedora-avocado.docker
>>>>>>  create mode 100755 tests/docker/test-avocado
>>>>>>
>>>>>> diff --git a/tests/docker/Makefile.include 
>>>>>> b/tests/docker/Makefile.include
>>>>>> index ef1a3e62eb..0e3d108dde 100644
>>>>>> --- a/tests/docker/Makefile.include
>>>>>> +++ b/tests/docker/Makefile.include
>>>>>> @@ -60,6 +60,7 @@ docker-image-debian-ppc64el-cross: docker-image-debian9
>>>>>>  docker-image-debian-s390x-cross: docker-image-debian9
>>>>>>  docker-image-debian-win32-cross: docker-image-debian8-mxe
>>>>>>  docker-image-debian-win64-cross: docker-image-debian8-mxe
>>>>>> +docker-image-fedora-avocado: docker-image-fedora
>>>>>>  docker-image-travis: NOUSER=1
>>>>>>
>>>>>>  # Expand all the pre-requistes for each docker image and test 
>>>>>> combination
>>>>>> diff --git a/tests/docker/dockerfiles/fedora-avocado.docker 
>>>>>> b/tests/docker/dockerfiles/fedora-avocado.docker
>>>>>> new file mode 100644
>>>>>> index 0000000000..55b19eebbf
>>>>>> --- /dev/null
>>>>>> +++ b/tests/docker/dockerfiles/fedora-avocado.docker
>>>>>> @@ -0,0 +1,25 @@
>>>>>> +FROM qemu:fedora
>>>>>> +
>>>>>> +ENV PACKAGES \
>>>>>> +    libvirt-devel \
>>>>>> +    nc \
>>>>>> +    python-avocado \
>>
>> python-avocado is a "Provides" of python2-avocado, but that may may go
>> away soon (as soon as Fedora 29).  I'd recommend using python2-avocado,
>> which is the proper package name available on Fedora.
>>
>>>>>> +    python2-devel python3-devel \
>>>>>> +    qemu-kvm \
>>>>>> +    tcpdump \
>>>>>> +    xz
>>>>>> +ENV PIP_PACKAGES \
>>>>>> +    avocado-qemu \
>>
>> +CC: Amador Pahim
>>
>> This pip package was created by Amador Pahim, but was intended for demo
>> purposes only.  Upon finding about this, he (cc'ed here) said that the
>> package is going to be removed to avoid further confusion.
>>
>> BTW, the intention is to have this supporting code for QEMU tests in the
>> QEMU tree itself.  Myself and Eduardo are working on a new version of
>> that supporting code.  Also, based on what I see on the test-avocado
>> file, this code is not needed at this point.
>>
>>>>>> +    avocado-framework-plugin-runner-remote \
>>>>>> +    avocado-framework-plugin-runner-vm \
>>
>> These two plugins are only needed if you intend to run tests on remote
>> machines.  Given "test-avocado" is not making use of those features,
>> there's no need to install them.
>>
>> As a side note, plugins are also available on Fedora as RPM packages, so
>> if I failed to understand the requirements for these plugins here, they
>> could still be installed as the python2-avocado-plugins-runner-remote
>> and python2-avocado-plugins-runner-vm RPM packages.
>>
>>>>>> +    avocado-framework-plugin-vt
>>>>>> +
>>
>> +CC: Lucas Rodrigues
>>
>> This is a pip package created by Lucas Rodrigues, and this code is
>> certainly used in the "test-avocado" file.  He told me he's using it to
>> run Avocado-VT tests in a venv on Ubuntu, Amazon Linux, etc, and he's
>> willing to maintain and update it.
>
> I'd suggest to look into virtual environments to run avocado-vt.
> virtual environments are more lightweight than using a docker
> container to run things and achieve similar functionality. Since I
> mostly never install python libraries and projects system wide these
> days, I didn't test this install mode in a docker container. Here's
> what I'd do on my Ubuntu 16.04 development laptop:
>
> $ virtualenv avocado_vt_demo
> Running virtualenv with interpreter /usr/bin/python2
> New python executable in /home/lmr/avocado_vt_demo/bin/python2
> Also creating executable in /home/lmr/avocado_vt_demo/bin/python
> Installing setuptools, pkg_resources, pip, wheel...done.
>
> $ source avocado_vt_demo/bin/activate
>
> (avocado_vt_demo) $ pip install avocado-framework-plugin-vt
> ... output trimmed ...
> Successfully built avocado-framework-plugin-vt autotest aexpect simplejson
> Installing collected packages: pbr, six, stevedore, avocado-framework,
> autotest, aexpect, simplejson, netaddr, netifaces,
> avocado-framework-plugin-vt
> Successfully installed aexpect-1.4.0 autotest-0.16.4
> avocado-framework-61.0 avocado-framework-plugin-vt-57.4 netaddr-0.7.19
> netifaces-0.10.7 pbr-4.0.3 simplejson-3.15.0 six-1.11.0
> stevedore-1.28.0
>
> (avocado_vt_demo) $ avocado vt-bootstrap --yes-to-all --vt-type qemu
> Failed to load plugin from module "avocado_vt.plugins.vt_list":
> EnvironmentError("Bootstrap missing. Execute 'avocado vt-bootstrap' or
> disable this plugin to get rid of this message",)
> Failed to load plugin from module "avocado_vt.plugins.vt":
> EnvironmentError("Bootstrap missing. Execute 'avocado vt-bootstrap' or
> disable this plugin to get rid of this message",)
> Running bootstrap for qemu
>
> 1 - Checking the mandatory programs and headers
> /usr/bin/xz OK
> /usr/sbin/tcpdump OK
> /bin/nc OK
> /bin/ip OK
> /usr/bin/arping OK
> /usr/bin/gcc OK
> /usr/include/python2.7/Python.h OK
> /usr/include/linux/types.h OK
> /usr/include/linux/unistd.h OK
> /usr/include/linux/socket.h OK
>
> ... output trimmed ...
>
> 9 - Checking for modules kvm, kvm-intel
> Module kvm loaded
> Module kvm-intel loaded
>
> VT-BOOTSTRAP FINISHED
> You may take a look at the following online docs for more info:
>  - http://avocado-vt.readthedocs.org/
>  - http://avocado-framework.readthedocs.org/
>
> (avocado_vt_demo) $ avocado run boot
> JOB ID     : 2ffba3965cd327e768bd8dadf4f82d6416a4f293
> JOB LOG    : 
> /home/lmr/avocado/job-results/job-2018-05-22T22.09-2ffba39/job.log
>  (1/1) io-github-autotest-qemu.boot: PASS (31.31 s)
> RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
> | CANCEL 0
> JOB TIME   : 33.25 s
>
> I can maintain the pip package mostly in sync with the other avocado
> releases. Just keep in mind that my bandwidth for maintenance tasks
> outside of my job @Amazon is limited, so you will have to nudge me if
> you need something urgently.
>
> Cheers,
>
> Lucas
>
>>>>>> +ENV QEMU_CONFIGURE_OPTS --python=/usr/bin/python3
>>
>> OK, "python3-avocado" is not yet available on Fedora, because it still
>> tracks the 52.x (LTS) series.  So, if Python 3 is the version chosen, it
>> maybe a good idea to remove "python-avocado" from "PACKAGES" completely
>> and use "avocado-framework" instead on "PIP_PACKAGES".
>>
>>>>>
>>>>> I think this is inherited from qemu:fedora, no?
>>>>
>>>> It is, I have removed it.
>>>>
>>>>>
>>>>>> +
>>>>>> +RUN dnf install -y $PACKAGES
>>>>>> +RUN pip install $PIP_PACKAGES
>>>>>> +RUN avocado vt-bootstrap --yes-to-all --vt-type qemu
>>>>>> +
>>>>>> +RUN rpm -q $PACKAGES | sort > /packages.txt
>>>>>
>>>>> Can you keep the parent image's list with ">>" or appending to the old 
>>>>> $PACKAGES
>>>>> in the above "ENV" directive?
>>>>>
>>>>>> +
>>>>>> +ENV FEATURES mingw clang pyyaml asan avocado
>>>>>
>>>>> Similarly, is it possible to append to the parent list instead of 
>>>>> overriding?
>>>>
>>>> I have changed both of these to append to the original variables.
>>>
>>> Feel free to add my R-b then:
>>> Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
>>>
>>> I'll wait your respin to try again, hoping I got my proxy issues solved.
>>>
>>>>>
>>>>>> diff --git a/tests/docker/test-avocado b/tests/docker/test-avocado
>>>>>> new file mode 100755
>>>>>> index 0000000000..40474db2ce
>>>>>> --- /dev/null
>>>>>> +++ b/tests/docker/test-avocado
>>>>>> @@ -0,0 +1,28 @@
>>>>>> +#!/bin/bash -e
>>>>>> +#
>>>>>> +# Avocado tests on Fedora, as these are a real pain on Debian systems
>>>>>
>>>>> Shouldn't pip packages work just well on Debian too? What are the pain?
>>>>> (Cc'ing Cleber who may want to know this).
>>>>
>>>> There is no debian package at the moment.
>>>>
>>
>> That's why Fam said *pip* packages.  In my experience with pip packages
>> on Debian:
>>
>>  * avocado-framework: installation and general usage works well.  Found
>> one issue though when running on containers, though, with an easy
>> workaround[1]
>>
>>  * avocado-framework-plugins-vt: currently at version 57.x, and when
>> used outside of venvs, doesn't get the "test-providers" directory
>> right[2].  Same happened with latest master (using pip install -e
>> git+https://github.com/avocado-framework/avocado-vt.git#egg=avocado-plugins-vt)
>>
>> [1] -
>> https://trello.com/c/yEH8ixps/1328-avocado-crashes-on-systems-without-less
>>
>> [2] -
>> https://trello.com/c/0BZZE4CR/1329-avocado-vt-bootstrap-from-pip-package-fails-to-find-test-providers-dir
>>
>> - Cleber.
>>
>>>> Alistair
>>>>
>>>>>
>>>>> Fam
>>>>>
>>>>>> +#
>>>>>> +# Copyright (c) 2018 Western Digital.
>>>>>> +#
>>>>>> +# Authors:
>>>>>> +#  Alistair Francis <address@hidden>
>>>>>> +#
>>>>>> +# This work is licensed under the terms of the GNU GPL, version 2
>>>>>> +# or (at your option) any later version. See the COPYING file in
>>>>>> +# the top-level directory.
>>>>>> +#
>>>>>> +# Run this test: NOUSER=1 make address@hidden
>>>>>> +
>>>>>> +. common.rc
>>>>>> +
>>>>>> +requires avocado
>>>>>> +
>>>>>> +cd "$BUILD_DIR"
>>>>>> +
>>>>>> +DEF_TARGET_LIST="x86_64-softmmu"
>>>>>> +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
>>>>>> +build_qemu
>>>>>> +install_qemu
>>>>>> +
>>>>>> +export PATH="${PATH}:$(pwd)"
>>>>>> +avocado run boot --vt-qemu-bin ./x86_64-softmmu/qemu-system-x86_64
>>>>>> --
>>>>>> 2.17.0
>>>>>>
>>>
>
>
>
> --
> Lucas



-- 
Lucas