Re: [RFC PATCH] gitlab: add a binary build to project registry

[Daniel P . Berrangé]

On Wed, Jun 22, 2022 at 03:43:54PM +0100, Alex Bennée wrote:
> 
> Peter Maydell <peter.maydell@linaro.org> writes:
> 
> > On Wed, 22 Jun 2022 at 13:33, Alex Bennée <alex.bennee@linaro.org> wrote:
> >>
> >> There have been requests from other projects such LKFT to have
> >> "official" docker images with pre-built QEMU binaries. These could
> >> then be consumed by downstream CI systems by pulling directly from the
> >> qemu-project container registry. The final image could then be run by
> >> doing:
> >>
> >>   docker run --rm -it \
> >>     registry.gitlab.com/qemu-project/qemu/qemu/debian-amd64-binaries \
> >>     /opt/bin/qemu-system-aarch64 $ARGS
> >>
> >> To keep the build time down we skip user-mode, documents, plugins and
> >> a selection of the more esoteric hardware emulations. Even so it still
> >> takes over an hour to build and install everything.
> >
> >> This is very much an RFC for now as we have traditionally not provided
> >> binaries for our users except for the slightly special case of
> >> Windows. As currently structured this will rebuild the binaries on
> >> every merge but we could change the generation rules to only trigger
> >> for tagged or stable branch pushes. We also wouldn't be testing these
> >> binaries so we are basically assuming they are OK by the nature of
> >> being built from master which in theory never breaks.
> >
> > I'm a bit uncertain about providing "official" docker images
> > or other pre-built binaries, because it might give the impression
> > that these are OK to use with KVM, when in fact they won't necessarily
> > get security fixes in a timely manner, and almost all users of QEMU for
> > KVM purposes are better off with the distro QEMU.
> 
> Do we have any sort of idea how long it takes from a CVE being fixed in
> a distro kernel to the eventual merge of a patch in master?
> 
> Anyway the main use case for this is emulation where we want to get new
> features under -cpu max into the CI loops as soon as possible. I think
> the LKFT guys are testing KVM as well though and again want to see new
> KVM features as soon as possible.
> 
> I'm not proposing these images are uploaded to docker.io so less likely
> to be hit by the default:
> 
>   docker run qemu:latest
> 
> (they instead get 6 year old things packages by someone calling
> themselves qemu: https://hub.docker.com/u/qemu)

I guess I would question whether we really need to introduce ourselves
as a dependancy of arbitrary downstream projects. We will have little
insight into what other projects are doing with any containers we publish.
This in turn means our scope for making changes to the containers over
time is quite limited, unless we're willing to risk breaking those
unknown projects' CI systems.

Personally, I feel CI systems are already fragile enough that it is
better to have a self contained system as much as possible. This is why,
for example, libvirt's integration test suite builds QEMU from git
itself. We get exact control over which QEMU features are built, and
get to choose which distros we target too. It isn't hard for apps wanting
to use QEMU to perform a build:

  
https://gitlab.com/libvirt/libvirt/-/blob/master/ci/integration-template.yml#L5

Probably the hardest part here is understanding the build time package
dependancies that are required to be installed, if you want to all QEMU
features enabled. We could perhaps use lcitool to record the full set of
build dependancies for each distro in git, so 3rd party CI can grab the
package list and install it, and thus get the trivial build done. Even
better if we publish the package list info in our docs too.

I wonder why LKFT isn't building QEMU themselves in the way that they
need already ?  Would be nice to know what their problems are, rather
than just assume that a container build is the solution.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|