Re: [RFC PATCH 2/2] GitLab CI: crude mapping of PMM's scripts to jobs

[Cleber Rosa]

On Mon, Feb 03, 2020 at 03:36:28PM -0200, Wainer dos Santos Moschetta wrote:
> Hi Cleber,
> 
> On 2/3/20 1:23 AM, Cleber Rosa wrote:
> > This is a crude and straightforward mapping of Peter's
> > "remake-merge-builds" and "pull-buildtest" scripts.
> > 
> > Some characteristics were removed for simplicity sake (but eventually
> > will), including:
> >   * number of simultaneous make jobs
> >   * make's synchronous output, not needed because of previous point
> >   * out-of-tree builds
> > 
> > This covers the "x86-64 Linux with a variety of different build
> > configs"[1].  I've personally tested all of them, and only had
> > issues with the "notcg" job[2], but it seems to be a test specific
> > issue with the nested KVM I was using.
> 
> 
> Could you put a comment in the commit text or in-code explaining why it
> builds QEMU with  --disable-libssh on all the jobs?
>

Yes, sure.  Answering it now, it's because of
https://bugs.launchpad.net/qemu/+bug/1838763, which was present
in tests/docker/dockerfiles/ubuntu1804.docker, and that I've hit
when I tried to be stubborn and ignore it. :)

> > 
> > [1] - https://wiki.qemu.org/Requirements/GatingCI#Current_Tests
> > [2] - https://paste.centos.org/view/1dd43a1c
> > 
> > Signed-off-by: Cleber Rosa <address@hidden>
> > ---
> >   .gitlab-ci.yml | 116 +++++++++++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 116 insertions(+)
> > 
> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > index d2c7d2198e..eb4077e2ab 100644
> > --- a/.gitlab-ci.yml
> > +++ b/.gitlab-ci.yml
> > @@ -2,6 +2,8 @@ include:
> >     - local: '/.gitlab-ci-edk2.yml'
> >   build-system1:
> > + rules:
> > + - if: '$CI_COMMIT_REF_NAME != "staging"'
> >    before_script: &before_scr_apt
> >    - apt-get update -qq
> >    - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev 
> > genisoimage
> > @@ -17,6 +19,8 @@ build-system1:
> >    - make -j2 check
> >   build-system2:
> > + rules:
> > + - if: '$CI_COMMIT_REF_NAME != "staging"'
> >    before_script:
> >     *before_scr_apt
> >    script:
> > @@ -31,6 +35,8 @@ build-system2:
> >    - make -j2 check
> >   build-disabled:
> > + rules:
> > + - if: '$CI_COMMIT_REF_NAME != "staging"'
> >    before_script:
> >     *before_scr_apt
> >    script:
> > @@ -47,6 +53,8 @@ build-disabled:
> >    - make -j2 check-qtest SPEED=slow
> >   build-tcg-disabled:
> > + rules:
> > + - if: '$CI_COMMIT_REF_NAME != "staging"'
> >    before_script:
> >     *before_scr_apt
> >    script:
> > @@ -67,6 +75,8 @@ build-tcg-disabled:
> >               248 250 254 255 256
> >   build-user:
> > + rules:
> > + - if: '$CI_COMMIT_REF_NAME != "staging"'
> >    before_script:
> >     *before_scr_apt
> >    script:
> > @@ -78,6 +88,8 @@ build-user:
> >    - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user
> >   build-clang:
> > + rules:
> > + - if: '$CI_COMMIT_REF_NAME != "staging"'
> >    before_script:
> >     *before_scr_apt
> >    script:
> > @@ -92,6 +104,8 @@ build-clang:
> >    - make -j2 check
> >   build-tci:
> > + rules:
> > + - if: '$CI_COMMIT_REF_NAME != "staging"'
> >    before_script:
> >     *before_scr_apt
> >    script:
> > @@ -111,3 +125,105 @@ build-tci:
> >    - QTEST_QEMU_BINARY="x86_64-softmmu/qemu-system-x86_64" 
> > ./tests/qtest/pxe-test
> >    - QTEST_QEMU_BINARY="s390x-softmmu/qemu-system-s390x"
> >      ./tests/qtest/pxe-test -m slow
> > +
> > +ubuntu-18.04.3-x86_64-notools:
> > + tags:
> > + - ubuntu_18.04.3
> > + - x86_64
> > + rules:
> > + - if: '$CI_COMMIT_REF_NAME == "staging"'
> > + script:
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/remake-merge-builds#n22
> > + - ./configure --target-list=arm-softmmu --disable-tools --disable-libssh
> > + # There is no make / make check in the "pull-buildtest" script for this.
> > + # Question: should it at least be built? Or dropped?
> > + - make
> > +
> > +ubuntu-18.04.3-x86_64-all-linux-static:
> 
> Doesn't it need to LD_PRELOAD on this runner too? ->
> 
> |https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/pull-buildtest#n24
> |
> 
> > + tags:
> > + - ubuntu_18.04.3
> > + - x86_64
> > + rules:
> > + - if: '$CI_COMMIT_REF_NAME == "staging"'
> > + script:
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/remake-merge-builds#n25
> > + - ./configure --enable-debug --static --disable-system 
> > --disable-glusterfs --disable-libssh
> 
> Shouldn't it --disable-gnutls instead of --disable-glusterfs?
>

Actually, `--disable-glusterfs` was intended because I couldn't find
static versions of the glusterfs libs on that system (or maybe they're
not available anywhere).  That should be documented though.

Also, the build and make steps worked fine without `--disable-gnutls`.
I was trying to not carry over any "exception" type of arguments,
unless they proved to be necessary.  Maybe Peter can give some more
info about this (should it be kept or not)?

> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/pull-buildtest#n36
> > + - make
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/pull-buildtest#n45
> > + - make check V=1
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/pull-buildtest#n48
> > + - make check-tcg V=1
> 
> 
> Any special reason to split it in 3 steps instead of a single `make check
> check-tcg`?
> 
> That pattern continues on next jobs...
> 
>

I think it's simply because it's easier to track the command that
failed on the GitLab CI output.

> > +
> > +ubuntu-18.04.3-x86_64-all:
> > + tags:
> > + - ubuntu_18.04.3
> > + - x86_64
> > + rules:
> > + - if: '$CI_COMMIT_REF_NAME == "staging"'
> > + script:
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/remake-merge-builds#n26
> > + - ./configure --disable-libssh
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/pull-buildtest#n28
> > + - make
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/pull-buildtest#n37
> > + - make check V=1
> > +
> > +ubuntu-18.04.3-x86_64-alldbg:
> > + tags:
> > + - ubuntu_18.04.3
> > + - x86_64
> > + rules:
> > + - if: '$CI_COMMIT_REF_NAME == "staging"'
> > + script:
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/remake-merge-builds#n27
> > + - ./configure --disable-libssh
> Missing --enable-debug, right?
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/pull-buildtest#n27
> > + - make clean
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/pull-buildtest#n29
> > + - make
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/pull-buildtest#n37
> > + - make check V=1
> > +
> > +ubuntu-18.04.3-x86_64-clang:
> > + tags:
> > + - ubuntu_18.04.3
> > + - x86_64
> > + rules:
> > + - if: '$CI_COMMIT_REF_NAME == "staging"'
> > + script:
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/remake-merge-builds#n31
> > + - ./configure --disable-libssh --cc=clang --cxx=clang++ --enable-gtk 
> > --extra-cflags='-fsanitize=undefined -fno-sanitize=shift-base -Werror'
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/pull-buildtest#n33
> > + - make
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/pull-buildtest#n39
> > + - make check V=1
> > +
> > +ubuntu-18.04.3-x86_64-tci:
> > + tags:
> > + - ubuntu_18.04.3
> > + - x86_64
> > + rules:
> > + - if: '$CI_COMMIT_REF_NAME == "staging"'
> > + script:
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/remake-merge-builds#n33
> > + - ./configure --disable-libssh --enable-debug --enable-tcg-interpreter
> > + # 
> > https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/pull-buildtest#n34
> > + - make
> 
> Would make sense to add a commented `make check` entry as well?
> 
> https://git.linaro.org/people/peter.maydell/misc-scripts.git/tree/pull-buildtest#n40
>

Not sure about that, but surely every one of those exceptions need to
be tracked.

- Cleber.

signature.asc
Description: PGP signature