Re: Separate build rules (was Re: [Mingw-cross-env-list] Mingw64)

[Tony Theodore]

On 19 December 2010 23:54, Volker Grabsch <address@hidden> wrote:
> Tony Theodore schrieb:
>> On 19 December 2010 03:16, Tony Theodore <address@hidden> wrote:
>>
>> > Having said that, I thought I'd go ahead with a proof-of-concept,
>> > changesets attached.
>>
>> I've progressed to building all of mingw-cross-env using the mingw.org
>> runtime, and qt using mingw-w64. I've set up a repository here:
>>
>> https://bitbucket.org/tonyt/mce-multi-target/
>
> Thanks for your work. As soon as I find some time, I'll try to
> provide the necessary infrastructure in mingw-cross-env, possibly
> inspired by your fork.

based on your design...

> The idea is that in the near future, the fork will only differ
> from the main project by some additional *_BUILD_x86_64 sections.

I've no intention of maintaing a fork, this was really just a POC
exercise, and as  such I'm very happy with it. Many technical
learnings and it wraps up over a year of experiments. The
possibilities of multiple toolchain/targets are interesting (clang
based toolchain anyone?), and Make has a level of abstraction that
reminds me of LISP.

I'm still not convinced of the "need" for 64-bit builds, or if we have
the resources to support it. We certainly "can" run a stable 32-bit
mingw.org along with an experimental 64-bit mingw-w64. The main
problem with this approach is that investigating issues is a
confluence of 32/64-bit issues and differences in the run-times (along
with our normal cross-compile issues). I didn't test the 32-bit
mingw-w64.

Having said that, some numbers:

# packages (src/*.mk files): 164 (includes download-only packages)
# successful 32-bit builds : 160
# successful 64-bit builds : 131

This is only testing the build, the only test program I've tried
executing is  qt (and separately built the demos - which are on par
with the 32-bit build).

Quite a few of the failing packages are related to pthreads - the
mingw64 project uses it's own patches against cvs which conflict with
our autostatic patch (a modified version of this is in pthreads cvs).
This is the only scenario that doesn't fit nicely into the "common"
infrastructure, but I don't think it warrants trying to have separate
patch files.

Build times are basically the same across the different targets, the
only package that varies materially is gcc, as it also builds the crt
in mingw-w64 (~10 mins vs ~14 mins).

This might look like it's very close in terms of the number of
packages (over 80%), but I've found that each package takes an
inordinate amount of time (for the few I've been able to resolve).
PostgreSQL is the best example; 64-bit compatible for a very long
time, and I use the project's MSVC 64-bit binaries every day (since v9
was released). And yet, I can't get the mingw-w64 build past
configure!

It's really a question for the project, adding the infrastructure is a
worthwhile (and do-able) exercise, but the why/who of 64-bit support
remains open.

>> 1. Build order can be a little confusing. It initially looks like it's
>> building i686 targets, then switches to x86_64 after the first
>> specified package is reached. If you're calling "make package" this is
>> all fine, but a plain "make" will build atk_i686 and all i686 deps,
>> then switch.
>
> I think you can influence this by the order in which the macros
> generate the build rules, i.e. the order of the foreach loops.

I thought the same and put TARGETS in the outer loop. However, this
only determines the "definition" order (possibly, the output of make
-qp is bizarre). The execution order is determined by dependency
rules. I thought about creating some .PHONY: (TARGET)-PKG rules, but
that was more confusing than the build order.

>> 3. There's lot's of flexibility with consolidating common build
>> sections. freetds, openssl, and qt are examples of "good?" style, gcc
>> probably isn't. We'd want to establish some conventions around this as
>> you can do some elaborate and complex things.
>
> Feel free to provide any coding style improvements directly to
> mingw-cross-env. In general, I think it's a good idea for a fork
> not to drift apart too much.

We can revisit this if the infrastructure is implemented. I mentioned
it as I came up with several conflicting styles of build rules as I
went along, and was looking for some feedback. The $(subst
mingw,mingw64, ...) construct is an example of something that seems
nice and simple, but could be fragile in practice (eg. neglecting the
space after mingw will cause mingw-cross-env directory paths to be
modified).

>> 4. Still need to decide what to do with "empty" build rules, and
>> possibly find a way of distinguishing between a know failure and
>> package that simply downloads a tarball.
>
> I think this should be done by not providing those build rules
> at all. That is, and empty build rule means it is a download
> package. A missing build rule (*_BUILD_x86_64 not defined) means
> that the package is not (yet?) supported for 64-bit systems.

At the moment, empty build rules are much the same as missing build
rules - they both write the install marker, but the empty rule unpacks
the tarball and deletes the tmp-* directory.

> The only remaining question is what to do with packages that are
> by design 32-bit only (or 64-bit only, for that matter). An empty
> build rule would mean those are download-only packages, which would
> simply be wrong: Their downloads aren't useful at all for the other
> platform. In other hand, providing no build rule might suggest those
> would become available for the 64-bit platform in the near future.
> At the moment I think the latter is the lesser evil. There's nothing
> wrong with stating e.g. that the GCC package in unavailable for
> the 64-bit platform as well as stating that the mingw-w64 package
> is unvailable for the 32-bit platform. However, I'm open to better
> suggestions as long at those don't complicate things.

Having no build rule at all works well in practice (with my simple
implementation). It's only some toolchain packages that are
download-only and/or inapplicable to another target - and they are
very easy to identify. For normal cross-compiled packages, the
"missing" build rule clearly denotes a package that isn't (yet)
supported.

>> At some stage, we might want
>> to parse the src/*.mk files and update the package list in the docs
>> with their 64-bit support.
>
> That's a good idea. I think we should simply add one column for
> each architecture (i686, x86_64), stating whether this is supported
> or not.

I thought this might be one of the more interesting parts, but it's
way beyond my sed skills!

Hope you had a chance to take a holiday, and best wishes for the new year.

Cheers,

Tony