Re: [PATCH v2 04/13] tests: Update our CI to use CentOS Stream 9 instead of 8

[Daniel P . Berrangé]

On Fri, Apr 12, 2024 at 03:24:06PM +0200, Thomas Huth wrote:
> RHEL 9 (and thus also the derivatives) are available since two years
> now, so according to QEMU's support policy, we can drop the active
> support for the previous major version 8 now.
> Thus upgrade our CentOS Stream container to major version 9 now.

The second reason for doing this is that Centos Stream 8
will go EOL in about 1 month:

https://blog.centos.org/2023/04/end-dates-are-coming-for-centos-stream-8-and-centos-linux-7/

  "After May 31, 2024, CentOS Stream 8 will be archived
   and no further updates will be provided."

I'm seeking confirmation, but I suspect after that date we
will be unable to build centos8 containers, as the package
repos will likely be archived.

RHEL-8 and other derivatives (Alma Linux, Rocky Linux,
etc) remain actively supported by their respective vendors
/ communities. Only CentOS Stream EOLs.


This has implications for our CI on stable branches. It is
valid for our stable branches to continue targetting the
RHEL-8 family of distros, as a 2 year cutoff in our support
policy is evaluated at time of each given major release.

IOW, cherry-picking this change to switch to CentOS Stream
9 is possibly inappropriate for stable branches.

lcitool supports Alma Linux as target, so we could switch
stable branches to Alma Linux 8 if desired to keep CI
coverage of RHEL-8 family.

Thoughts ?

> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  .gitlab-ci.d/buildtest.yml                    | 16 ++++-----
>  .gitlab-ci.d/container-core.yml               |  4 +--
>  .../{centos8.docker => centos9.docker}        | 34 +++++++------------
>  tests/lcitool/mappings.yml                    | 20 -----------
>  tests/lcitool/refresh                         |  2 +-
>  tests/vm/centos                               |  4 +--
>  6 files changed, 26 insertions(+), 54 deletions(-)
>  rename tests/docker/dockerfiles/{centos8.docker => centos9.docker} (82%)
> 
> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
> index cfdff175c3..9f34c650d6 100644
> --- a/.gitlab-ci.d/buildtest.yml
> +++ b/.gitlab-ci.d/buildtest.yml
> @@ -158,9 +158,9 @@ build-system-centos:
>      - .native_build_job_template
>      - .native_build_artifact_template
>    needs:
> -    job: amd64-centos8-container
> +    job: amd64-centos9-container
>    variables:
> -    IMAGE: centos8
> +    IMAGE: centos9
>      CONFIGURE_ARGS: --disable-nettle --enable-gcrypt 
> --enable-vfio-user-server
>        --enable-modules --enable-trace-backends=dtrace --enable-docs
>      TARGETS: ppc64-softmmu or1k-softmmu s390x-softmmu
> @@ -242,7 +242,7 @@ check-system-centos:
>      - job: build-system-centos
>        artifacts: true
>    variables:
> -    IMAGE: centos8
> +    IMAGE: centos9
>      MAKE_CHECK_ARGS: check
>  
>  avocado-system-centos:
> @@ -251,7 +251,7 @@ avocado-system-centos:
>      - job: build-system-centos
>        artifacts: true
>    variables:
> -    IMAGE: centos8
> +    IMAGE: centos9
>      MAKE_CHECK_ARGS: check-avocado
>      AVOCADO_TAGS: arch:ppc64 arch:or1k arch:s390x arch:x86_64 arch:rx
>        arch:sh4 arch:nios2
> @@ -327,9 +327,9 @@ avocado-system-flaky:
>  build-tcg-disabled:
>    extends: .native_build_job_template
>    needs:
> -    job: amd64-centos8-container
> +    job: amd64-centos9-container
>    variables:
> -    IMAGE: centos8
> +    IMAGE: centos9
>    script:
>      - mkdir build
>      - cd build
> @@ -651,9 +651,9 @@ build-tci:
>  build-without-defaults:
>    extends: .native_build_job_template
>    needs:
> -    job: amd64-centos8-container
> +    job: amd64-centos9-container
>    variables:
> -    IMAGE: centos8
> +    IMAGE: centos9
>      CONFIGURE_ARGS:
>        --without-default-devices
>        --without-default-features
> diff --git a/.gitlab-ci.d/container-core.yml b/.gitlab-ci.d/container-core.yml
> index 08f8450fa1..5459447676 100644
> --- a/.gitlab-ci.d/container-core.yml
> +++ b/.gitlab-ci.d/container-core.yml
> @@ -1,10 +1,10 @@
>  include:
>    - local: '/.gitlab-ci.d/container-template.yml'
>  
> -amd64-centos8-container:
> +amd64-centos9-container:
>    extends: .container_job_template
>    variables:
> -    NAME: centos8
> +    NAME: centos9
>  
>  amd64-fedora-container:
>    extends: .container_job_template
> diff --git a/tests/docker/dockerfiles/centos8.docker 
> b/tests/docker/dockerfiles/centos9.docker
> similarity index 82%
> rename from tests/docker/dockerfiles/centos8.docker
> rename to tests/docker/dockerfiles/centos9.docker
> index ea618bf352..6cf47ce786 100644
> --- a/tests/docker/dockerfiles/centos8.docker
> +++ b/tests/docker/dockerfiles/centos9.docker
> @@ -1,15 +1,14 @@
>  # THIS FILE WAS AUTO-GENERATED
>  #
> -#  $ lcitool dockerfile --layers all centos-stream-8 qemu
> +#  $ lcitool dockerfile --layers all centos-stream-9 qemu
>  #
>  # https://gitlab.com/libvirt/libvirt-ci
>  
> -FROM quay.io/centos/centos:stream8
> +FROM quay.io/centos/centos:stream9
>  
>  RUN dnf distro-sync -y && \
>      dnf install 'dnf-command(config-manager)' -y && \
> -    dnf config-manager --set-enabled -y powertools && \
> -    dnf install -y centos-release-advanced-virtualization && \
> +    dnf config-manager --set-enabled -y crb && \
>      dnf install -y epel-release && \
>      dnf install -y epel-next-release && \
>      dnf install -y \
> @@ -42,7 +41,6 @@ RUN dnf distro-sync -y && \
>          glib2-static \
>          glibc-langpack-en \
>          glibc-static \
> -        glusterfs-api-devel \
>          gnutls-devel \
>          gtk3-devel \
>          hostname \
> @@ -82,6 +80,7 @@ RUN dnf distro-sync -y && \
>          lzo-devel \
>          make \
>          mesa-libgbm-devel \
> +        meson \
>          mtools \
>          ncurses-devel \
>          nettle-devel \
> @@ -95,25 +94,25 @@ RUN dnf distro-sync -y && \
>          pixman-devel \
>          pkgconfig \
>          pulseaudio-libs-devel \
> -        python38 \
> -        python38-PyYAML \
> -        python38-numpy \
> -        python38-pip \
> -        python38-setuptools \
> -        python38-wheel \
> +        python3 \
> +        python3-PyYAML \
> +        python3-numpy \
> +        python3-pillow \
> +        python3-pip \
> +        python3-sphinx \
> +        python3-sphinx_rtd_theme \
> +        python3-tomli \
>          rdma-core-devel \
>          sed \
>          snappy-devel \
>          socat \
>          spice-protocol \
> -        spice-server-devel \
>          swtpm \
>          systemd-devel \
>          systemtap-sdt-devel \
>          tar \
>          usbredir-devel \
>          util-linux \
> -        virglrenderer-devel \
>          vte291-devel \
>          which \
>          xfsprogs-devel \
> @@ -132,18 +131,11 @@ RUN dnf distro-sync -y && \
>      ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
>      ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
>  
> -RUN /usr/bin/pip3.8 install \
> -                    meson==0.63.2 \
> -                    pillow \
> -                    sphinx \
> -                    sphinx-rtd-theme \
> -                    tomli
> -
>  ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
>  ENV LANG "en_US.UTF-8"
>  ENV MAKE "/usr/bin/make"
>  ENV NINJA "/usr/bin/ninja"
> -ENV PYTHON "/usr/bin/python3.8"
> +ENV PYTHON "/usr/bin/python3"
>  # As a final step configure the user (if env is defined)
>  ARG USER
>  ARG UID
> diff --git a/tests/lcitool/mappings.yml b/tests/lcitool/mappings.yml
> index 407c03301b..03b974ad02 100644
> --- a/tests/lcitool/mappings.yml
> +++ b/tests/lcitool/mappings.yml
> @@ -1,66 +1,50 @@
>  mappings:
>    flake8:
> -    CentOSStream8:
>      OpenSUSELeap15:
>  
>    meson:
> -    CentOSStream8:
>      OpenSUSELeap15:
>  
>    python3:
> -    CentOSStream8: python38
>      OpenSUSELeap15: python311-base
>  
>    python3-PyYAML:
> -    CentOSStream8: python38-PyYAML
>      OpenSUSELeap15:
>  
>    python3-devel:
> -    CentOSStream8: python38-devel
>      OpenSUSELeap15: python311-devel
>  
>    python3-docutils:
> -    CentOSStream8:
>      OpenSUSELeap15:
>  
>    python3-numpy:
> -    CentOSStream8: python38-numpy
>      OpenSUSELeap15:
>  
>    python3-opencv:
> -    CentOSStream8:
>      OpenSUSELeap15:
>  
>    python3-pillow:
> -    CentOSStream8:
>      OpenSUSELeap15:
>  
>    python3-pip:
> -    CentOSStream8: python38-pip
>      OpenSUSELeap15: python311-pip
>  
>    python3-pillow:
> -    CentOSStream8:
>      OpenSUSELeap15:
>  
>    python3-selinux:
> -    CentOSStream8:
>      OpenSUSELeap15:
>  
>    python3-setuptools:
> -    CentOSStream8: python38-setuptools
>      OpenSUSELeap15: python311-setuptools
>  
>    python3-sphinx:
> -    CentOSStream8:
>      OpenSUSELeap15:
>  
>    python3-sphinx-rtd-theme:
> -    CentOSStream8:
>      OpenSUSELeap15:
>  
>    python3-sqlite3:
> -    CentOSStream8: python38
>      OpenSUSELeap15: python311
>  
>    python3-tomli:
> @@ -69,15 +53,11 @@ mappings:
>      Fedora:
>      Debian12:
>      OpenSUSELeap15:
> -    # Not available for Python 3.8
> -    CentOSStream8:
>  
>    python3-venv:
> -    CentOSStream8: python38
>      OpenSUSELeap15: python311-base
>  
>    python3-wheel:
> -    CentOSStream8: python38-wheel
>      OpenSUSELeap15: python311-pip
>  
>  pypi_mappings:
> diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
> index 692752a3df..24a735a3f2 100755
> --- a/tests/lcitool/refresh
> +++ b/tests/lcitool/refresh
> @@ -125,7 +125,7 @@ try:
>      # Standard native builds
>      #
>      generate_dockerfile("alpine", "alpine-318")
> -    generate_dockerfile("centos8", "centos-stream-8")
> +    generate_dockerfile("centos9", "centos-stream-9")
>      generate_dockerfile("debian", "debian-12",
>                          trailer="".join(debian12_extras))
>      generate_dockerfile("fedora", "fedora-38")
> diff --git a/tests/vm/centos b/tests/vm/centos
> index 097a9ca14d..d25c8f8b5b 100755
> --- a/tests/vm/centos
> +++ b/tests/vm/centos
> @@ -26,8 +26,8 @@ class CentosVM(basevm.BaseVM):
>          export SRC_ARCHIVE=/dev/vdb;
>          sudo chmod a+r $SRC_ARCHIVE;
>          tar -xf $SRC_ARCHIVE;
> -        make docker-test-block@centos8 {verbose} J={jobs} NETWORK=1;
> -        make docker-test-quick@centos8 {verbose} J={jobs} NETWORK=1;
> +        make docker-test-block@centos9 {verbose} J={jobs} NETWORK=1;
> +        make docker-test-quick@centos9 {verbose} J={jobs} NETWORK=1;
>      """
>  
>      def build_image(self, img):
> -- 
> 2.44.0
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|