Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Daniel P . Berrangé]

On Mon, Jun 20, 2022 at 10:29:14AM +0000, Zhang, Chen wrote:
> 
> 
> > -----Original Message-----
> > From: Thomas Huth <thuth@redhat.com>
> > Sent: Monday, June 20, 2022 5:44 PM
> > To: Zhang, Chen <chen.zhang@intel.com>; Daniel P. Berrangé
> > <berrange@redhat.com>
> > Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> > devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> > Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>; Markus
> > Armbruster <armbru@redhat.com>; Peter Maydell
> > <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>; Yuri
> > Benditovich <yuri.benditovich@daynix.com>; Andrew Melnychenko
> > <andrew@daynix.com>
> > Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a
> > submodule for QEMU
> > 
> > On 20/06/2022 11.29, Zhang, Chen wrote:
> > >
> > >
> > >> -----Original Message-----
> > >> From: Thomas Huth <thuth@redhat.com>
> > >> Sent: Monday, June 20, 2022 4:47 PM
> > >> To: Daniel P. Berrangé <berrange@redhat.com>; Zhang, Chen
> > >> <chen.zhang@intel.com>
> > >> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> > >> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> > >> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
> > Markus
> > >> Armbruster <armbru@redhat.com>; Peter Maydell
> > >> <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>; Yuri
> > >> Benditovich <yuri.benditovich@daynix.com>; Andrew Melnychenko
> > >> <andrew@daynix.com>
> > >> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as
> > >> a submodule for QEMU
> > >>
> > >> On 20/06/2022 10.11, Daniel P. Berrangé wrote:
> > >>> On Mon, Jun 20, 2022 at 05:59:06AM +0000, Zhang, Chen wrote:
> > >>>>
> > >>>>
> > >>>>> -----Original Message-----
> > >>>>> From: Daniel P. Berrangé <berrange@redhat.com>
> > >>>>> Sent: Friday, June 17, 2022 4:05 PM
> > >>>>> To: Zhang, Chen <chen.zhang@intel.com>
> > >>>>> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> > >>>>> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> > >>>>> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
> > >>>>> Markus Armbruster <armbru@redhat.com>; Peter Maydell
> > >>>>> <peter.maydell@linaro.org>; Thomas Huth <thuth@redhat.com>;
> > >> Laurent
> > >>>>> Vivier <lvivier@redhat.com>; Yuri Benditovich
> > >>>>> <yuri.benditovich@daynix.com>; Andrew Melnychenko
> > >>>>> <andrew@daynix.com>
> > >>>>> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project
> > >>>>> as a submodule for QEMU
> > >>>>>
> > >>>>> On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
> > >>>>>> Make iovisor/ubpf project be a git submodule for QEMU.
> > >>>>>> It will auto clone ubpf project when configure QEMU.
> > >>>>>
> > >>>>> I don't think we need todo this. As it is brand new functionality
> > >>>>> we don't have any back compat issues. We should just expect the
> > >>>>> distros to ship ubpf if they want their QEMU builds to take advantage
> > of it.
> > >>>>>
> > >>>>
> > >>>> Yes, agree. It's the best way to use the uBPF project.
> > >>>> But current status is distros(ubuntu, RHEL...) does not ship the
> > >>>> iovisor/ubpf like the iovisor/bcc. So I have to do it.
> > >>>> Or do you have any better suggestions?
> > >>>
> > >>> If distros want to support the functionality, they can add packages
> > >>> for it IMHO.
> > >>
> > >> Yes, let's please avoid new submodules. Submodules can sometimes be a
> > >> real PITA (e.g. if you forget to update before rsync'ing your code to
> > >> a machine that has limited internet access), and if users install
> > >> QEMU from sources, they can also install ubpf from sources, too.
> > >> And if distros want to support this feature, they can package ubpf on
> > >> their own, as Daniel said.
> > >
> > > Hi Daniel and Thomas,
> > >
> > > I don't know much the background history of QEMU submodules, but
> > meson
> > > build is a submodule for QEMU too. It means user can't install QEMU
> > > from sources with limited internet access.
> > 
> > There is no written policy, but I think the general consensus is that we 
> > only
> > ship code in submodules if:
> > 
> > 1) It's not available in a required version in distros yet
> > 
> > and
> > 
> > 2) it is essentially required to build QEMU (like meson) or if the feature 
> > has
> > been part of the QEMU sources before and then moved to a separate
> > repository (like slirp).
> > 
> > We ship meson as a submodule since we require some meson features that
> > are not available with the meson versions in the distros yet. Once the 
> > distros
> > catch up, we'll likely remove the meson submodule from QEMU.
> > 
> > > And back to Daniel's comments,  Yes, the best way is distros add the
> > > ubpf packages, But maybe it's too late to implement new features for
> > > us. We can introduce the submodule now and auto change to the distros's
> > lib when distros add it.  For example QEMU's submodule SLIRP do it in the
> > same way.
> > 
> > slirp used to be part of the QEMU repository, but then has been moved to a
> > separate project a while ago. However, at that point in time there weren't
> > any packages ins distros yet, so we had to include it as a submodule.
> > 
> > Now that the distros ship it, too, we're planning to remove the slirp
> > submodule from QEMU soon, see:
> > 
> >   https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00974.html
> > 
> > > It make user experience the latest technology with no other
> > > dependencies.
> > 
> > Well, that's only true if we update the submodule in QEMU regularly. If we
> > forget to update, we could easily miss some important (maybe even security
> > related) fixes from the upstream projects. This can be a nightmare for 
> > distros,
> > when they then have to go around and look into each and every projects
> > whether they embed a certain code module that needs a CVE fix. It's better
> > if it can be fixed in one central spot instead.
> > 
> > > uBPF infrastructure have the ability to extend the capabilities
> > > without requiring changing source code. If we not allow it, we have to
> > > re-implement all the eBPF assembler, disassembler, interpreter, and JIT
> > compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by
> > license issue).
> > 
> > Not sure whether I understood that statement right ... nobody said that
> > QEMU should not allow it - we just suggested to rely on a system 
> > installation
> > of ubpf instead of embedding the code. Or is that not possible?? (I don't
> > know that project yet - isn't it possible to compile it as a shared 
> > library?)
> 
> Thanks for your details explanation.
> It looks better to introduce the uBPF shared library for QEMU.
> For example:
> ./configure --ubpf-lib=path

I've not looked, so maybe it already does this, but ideally  'uBPF'
would ship a 'pkg-config'  file, so that apps can automatically find
it and set the right cflags/libs etc for the compiler. For configure
integration, normally we'd expect it to be --enable-ubpf/--disable-ubpf,
with it automatically enabling itself if the pkg-config file is found.
Take a look at handling of some existing libraries we depend on for
examples.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|