Re: [PATCH v6 01/10] gnu: cross: Use CROSS_*_INCLUDE_PATH for system headers.

[Jan Nieuwenhuizen]

Andy Wingo writes:

Hi!

[Manolis: I've written a bit about the cross setup below, do you want
 to have a look?]

> Mostly formatting nits, only a couple substantial comments.  Looking
> great.

Oh that's super, thanks for all your help!

It seems I have picked-up the convention of doing

   (if condition <simple-then>
       <longer-else>))

now that I think of it, could be lispy with optionally longer else
branch.  I have wondered why scheme/guile does not have that.  Possibly
it's an artifact of more imperative days?

>> +                     (if libc
>> +                         (setenv "CROSS_LIBRARY_PATH"
>> +                                 (string-append
>> +                                  libc "/lib"
>> +                                  ":" libc "/i686-w64-mingw32/lib")))
>
> Use "when" please

Yes, done.

>> +(define (native-libc target)
>> +  (if (mingw-target? target) mingw-w64
>> +      glibc))
>
> The if should be either all on one line, or on three lines

Changed to three lines; I expect this to get longer in time.

>> Subject: [PATCH 07/10] gnu: ncurses: support mingw.
>> +              (cond ((mingw-target? target)
>> +                  (with-directory-excursion (string-append out "/bin")
>> +                    (for-each
>> +                     (lambda (lib)
>
> Because the consequent is so large, please indent the condition on the
> line after "cond", i.e.
>
>   (cond
>    ((mingw-target? target)
>     ...))

Ok.

>
>> +++ b/gnu/packages/patches/readline-6.3-mingw.patch
>> @@ -0,0 +1,128 @@
>> +Mingw lacks some SIG*.  Taken from
>> +
>> + wget
>> https://raw.githubusercontent.com/Alexpux/MINGW-packages/master/mingw-w64-readline/readline-6.3-mingw.patch
>> +
>> +some updates to make it apply.
>> +
>> +Upstream status: Not presented to upstream.
>
> Probably needs to go upstream I think, and we should clarify the
> licensing on it or rewrite it.

Yes.  I have sent the owner of the github archive a mail for
clarification.

>> +                                        ,(if (mingw-target?) "/bin"
>> +                                             "/lib"))
>
> One line or three lines please

Sure, one line.

>> Subject: [PATCH 10/10] gnu: guile-2.0: support mingw.
>> +                    ,@(if (mingw-target?)
>> +                          `(("bash" ,bash)
>> +                            ("guile" ,guile-2.0))
>> +                          '())))
>
> Would we not need Guile always when cross-compiling?

Hmm.  I'm adding bash here and removing it below for mingw; which makes
some sense.  Guile should be included via self-native-input? -- ah, I
prolly unset self-native-input? while experimenting with a cross-guile
package and /there/ I needed this.  Retried without guile, removed,
changed to one line.

>> +             ,@(if (mingw-target?) '()
>> +                   `(("bash" ,bash)))))
>
> One line please.

Ok.

>> @@ -167,7 +172,11 @@ without requiring the source code to be rewritten.")
>>                    (let ((bash (assoc-ref inputs "bash")))
>>                      (substitute* "module/ice-9/popen.scm"
>>                        (("/bin/sh")
>> -                       (string-append bash "/bin/bash")))))
>> +                       ,(if (mingw-target?)
>> +                            "cmd.exe"
>> +                            `(if bash
>> +                                 (string-append bash "/bin/bash")
>> +                                 "bash"))))))
>>                  %standard-phases)))
>
> I guess the thing is, cmd.exe is part of the system, so it should be
> linked "dynamically" (i.e. via the PATH); but is cmd.exe really a bash
> replacement?  Is this the right way for open-pipe in Guile to work?

Good question.  I don't really know.  Probably it's not; but then, even
if we have bash would't you expect some utils like cat/grep/sed?  FWIW,
I spent two nights trying to port bash before I decided to give up, err
re-prioritise.  I'd love to provide that all, and I'd also love to get
some help with that ;-)

I can imagine that if I somehow installed a version of bash.exe
(msys/cygwin) in PATH, I would expect open-pipe to use that rather than
cmd.

I'm not sure if you have a suggestion for change here, or if we should
get more input first?

> Finally at the very end of all of this I think it would be great to have
> some kind of document, "how to compile software for windows using
> Guix".   WDYT?  That document could include any relevant details about
> the internal structure of a cross-compile -- how Guix links things
> together.  I know that for me, this information is verrrry easily paged
> out to long-term storage; takes a while to pull it back in :)

I think that's a good idea.  I spent quite some time figuring-out how
things work in Guix.  Most is just great; the biggest problems I
encountered were

   * any change made to cross-gcc / cross-libc triggers a rebuild of
     both worlds

and most changes I made were small, wrong and terribly difficult to
inspect.  I tried to work around that creating separate cross-gcc and
cross-libc functions...but that caused at least as much problems as it
solved.  I have no good solution for this...and

   * native packages (headers and libraries) were added to the cross
     build environment

which turned out to be an honest bug.  It led me around hacking
guix/build-system/gnu.scm: (standard-cross-packages) until I finally
found that (standard-packages) seemed to be the culprit, to eventually
find that the use of C_INCLUDE_PATH in bootstrap.scm needed to be
CROSS_CPATH...and when that worked we decided to go the other way and
use CROSS_C_INCLUDE_PATH etc.  Phew.

So, most of the difficulties I had should be fixed now; not sure what
difficulties the next person will have.  So, what to write exactly?

Below is a first attempt that I didn't want to send as a proper patch
yet.  I could do with some input, especially from Manolis.

Greetings,
Jan


@node Creating a New Cross Target
@section Creating a New Cross Target

As a first step of making a full port, you may want to start by creating
a cross target.  A cross target in essence is a cross compiler
@address@hidden<target>}}, which depends on
@address@hidden<target>}} a @address@hidden<target>}} and
possibly @address@hidden<target>}}.  Several cross targets
are available, such as @code{i586-pc-hurd}, @code{armhf-linux},
@code{avr}, @code{i686-w64-mingw32} and @code{mips64el-linux}.

Building a full gcc cross compiler depends on a c-library for the
target.  We can build a c-library for the target once we have a cross
compiler.  To break this loop a minimal gcc compiler can be built
without a c-library; we call this
@address@hidden<target>}}.  With this minimal gcc
compiler we cross compile the c-library and then we build the full cross
gcc.

In @code{gnu/packages/cross-base.scm} are functions to create these
cross packages.  Also, Guix needs to know the name of the dynamic
linker, see @var{glibc-dynamic-linker} in
@code{gnu/packages/bootstrap.scm}.

@menu
* Rebuilding My World::
* GCC and Cross Environment Paths::
* The MinGW Cross Target::
@end menu

@node Rebuilding My World
@subsection Rebuilding My World

Why is it that we all tend love to rebuild our world, yet like it
somewhat less when others decide do it for us?  One of the great things
of Guix is that it tracks all dependencies and will rebuild any package
that is out of date: We never have to worry that doing a fresh, clean
build does not reproduce.  However, if we make the tiniest change for
our cross build to the @var{ncurses} package (who named it so
appropriately?) and Guix will first rebuild all of our worlds before it
gives us the feedback on our modification.

What we can do is to create a temporary alternative package hierarchy.
We copy @var{ncurses} to @var{cross-ncurses}

@example
(define-public cross-ncurses
   @dots{}
   (name "cross-ncurses")
   @dots{})
@end example

and because we are really testing readline, we copy that too

@example
(define-public cross-readline
   @dots{}
   (name "cross-readline")
   @dots{}
   (propagated-inputs `(("ncurses" ,cross-ncurses)))
   @dots{})
@end example

which we then use in our copied @var{cross-guile} package.  When we are
satisfied with our change, we replace the original packages descriptions
with the @var{cross-} variants, remove the @var{cross-} prefix and
enjoy a favorite beverage while we have our worlds rebuild.

Because Guix uses pseudo cross-compilation in the bootstrap process
@ref{Bootstrapping}, changes to @var{cross-gcc} may rebuild our native
world and it could be helpful to work with copies of @var{cross-gcc} and
@var{cross-libc} too.

@node GCC and Cross Environment Paths
@subsection GCC and Cross Environment Paths

@c See <http://gcc.gnu.org/ml/gcc/2013-02/msg00124.html>
@c <http://bugs.gnu.org/22186> and
@c <https://lists.gnu.org/archive/html/guix-devel/2016-04/msg00533.html>
@c for a discussion of what follows.
Some build systems compile and run programs at build time to generate
host-specific data.  This means we usually have two compilers installed:
@code{gcc} and @code{<target>-gcc}.  Guix does not use
@file{/usr/include} and @file{/usr/lib} to install additional headers
and libraries, instead it adds to environment path variables such as
@var{C_INCLUDE_PATH} and @var{LIBRARY_PATH}.  To distinguish between
native build-time headers and libraries and cross-built target system
headers and libraries, we use a patched gcc as cross compiler.  The
cross compiler instead looks at @var{CROSS_C_INCLUDE_PATH} and
@var{CROSS_LIBRARY_PATH}.

@node The MinGW Cross Target
@subsection The MinGW Cross Target

The MinGW target is somewhat special in that it does not support
@var{glibc}.  Gcc needs to be passed the @code{--with-newlib} flag and
instead we use the combined c-library and free reïmplementation of
Windows kernel-headers and system-libraries provided by the MinGW-w64
project.  Also, it's not POSIX so it often needs explicit support,
sometimes we need to create a patch ourselves.

@example
$ guix build --target=i686-w64-mingw32 hello
@dots{}
/gnu/store/deadbeef123-hello-2.10
$ guix environment --ad-hoc wine
$ wine /gnu/store/deadbeef123-hello-2.10/bin/hello.exe
Hello, world!
@end example

-- 
Jan Nieuwenhuizen <address@hidden> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar®  http://AvatarAcademy.nl