Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Markus Armbruster]

Paolo Bonzini <address@hidden> writes:

> On 27/06/19 11:03, Daniel P. Berrangé wrote:
>> On Mon, Jun 10, 2019 at 01:14:57PM +0200, Paolo Bonzini wrote:
>>> The Meson build system is integrated in the existing configure/make steps
>>> by invoking Meson from the configure script and converting Meson's 
>>> build.ninja
>>> rules to an included Makefile.
>> 
>> Why did you take the route of converting ninja rules into makefile
>> rules, as opposed to just having some stub makefile rules which
>> directly invoke ninja where needed ?
>
> There are two parts of this.  One is practical and has to do with
> supporting a step-by-step transition.  Using ninja2make makes it trivial
> to have make build products that depend on meson build products, and
> this way bottom up is a natural direction to do the conversion, which is
> bottom up.  You'd start from libqemuutil.a and code generators (tracing
> + QAPI), then go to the tools and the emulators.

*If* the conversion is too big a task to permit doing it all at once,
then the step by step strategy you describe makes sense to me.

The trouble with step by step is running out of steam before the final
step.  That would leave us worse off.  Even an overly protracted
conversion would be bad.

Thus, my standard question on any proposed step-by-step conversion:
commitment to finishing it?  I'd be quite happy to take your word for
it.

> The second is a design decision that simplifying the Make/meson
> integration is *not* a goal.  Rather the goals are: 1) making the
> transition easier on developers; 2) avoiding magic in meson.build at all
> costs.  More specifically:
>
> - it should remain trivial to do things that used to be trivial, and
> most "make" invocations should be kept the same at least until
> everything is converted and we can perhaps declare a flag day.  People
> are used to "make check" or "make subdir-x86_64-softmmu", those should
> continue to work while the transition is in progress.
>
> - it should be possible to modify meson.build without knowing QEMU
> specific details, and that should be _already_ possible now at the
> beginning of the transition (to avoid creating technical debt).  This
> means keeping the magic confined in Makefile rules and external scripts,
> while having a pretty plain meson.build.

I wholeheartedly agree with (2).  Reducing magic is the whole point.

Your plan confines new magic to "Makefile rules and external scripts".
We'll get actual reduction only if we can retire or at least radically
simplify them at some point.

I'm more ambivalent on (1).  Yes, making the transition easier for
developers is worth hiding a certain amount of magic out of their way.
But not any amount.  We should try to preserve the targets printed by
"make help".

> I expect testing might also require some hand-holding, because "meson
> test" does not integrate with "make -j" and to keep supporting our "make
> check-*" targets.  However, until the make->ninja flag day we could
> generate tap-driver Makefile rules from "meson introspect --tests"
> output.  Basically I'm dropping Makefile magic in favor of build rule
> generators are written in high-level languages.

A PoC for selected tests would be nice.

>> Obviously this series is just some initial integration, but eventually
>> when we get a complete conversion, I think it will look pretty wierd
>> if we're still converting ninja to make.
>
> I agree; once all the build rules are converted the Makefile could be as
> simple as
>
>       all:
>       include config.mak
>       include tests/docker/Makefile.include
>       include tests/vm/Makefile.include
>       .NOTPARALLEL:
>       %:
>               ninja $@
>
>       subdir-%-softmmu:
>               ninja qemu-system-$*
>
>       subdir-%-linux-user:
>               ninja qemu-$*
>
>       check:
>               $(MESON) test
>
>       check-%:
>               $(MESON) test --suite $*
>
> etc. (and likewise the configure script could just translate the command
> line options before invoking meson).

I wouldn't bother keeping around such an empty shell of a Makefile.

Keeping configure as a wrapper could be marginally more useful, mostly
for people trying to build QEMU for the first time.

>                                       But for now, since rules are
> written half in meson and half in make, ninja2make seems the most
> transparent way to integrate the two.
>
>> Part of the downside of our current build system is that although it
>> uses make, the usage is wierd and QEMU specific structure. It would
>> be a shame to pick meson and then again use it is a way that is wierd
>> and QEMU specific.
>
> I agree, this is why it's important to have at least a standard meson.build.
>
> Some knowledge of config-host.mak is needed, because meson.build uses
> declare_dependency() instead of dependency() to link with libraries that
> were already found by configure, but that's it.

Ignorant question: could the switch to Meson enable doing less in
configure?  It's big and sloooow.