[bug#39309] [PATCH WIP] gnu: add stack.

[Timothy Sample]

Hi John,

John Soo <address@hidden> writes:

>> At this point, it might be worth joining the Nix folks on that issue,
>> and suggesting a patch to Cabal to call “hsc2hs” with deduplicated
>> flags.  It will help that the other deduplication code was written by
>> the same person who filed the bug you linked to earlier.  Once the patch
>> lands, we can apply it to our Cabal and then Stack will build.  Yay!
>
> Sorry I'm a bit confused by all this. I guess if we can rely on a
> patch into cabal that would be the least effort, right?

Exactly.  Also, sorry if I’m being a little unclear.  I’ve been kind of
thinking out loud with these messages, as you’ll see below.  ;)

> Is there already a patch to deduplicate flags passed to hsc2hs?  Do we
> need to comment anywhere or express support?

I couldn’t find a patch.  Expressing support was definitely what I was
suggesting.  I managed to find a note saying what I just said:

    https://github.com/NixOS/nixpkgs/issues/49206#issuecomment-470324743

I don’t think anything has become of it since then, though.

There is one other thing we could do, but I’m still thinking it through.
The reason we hit the limit is because of the way we use the
“extra-lib-dirs” and “extra-include-dirs” flags when configuring.  Every
time we pass Cabal a list of directories via these flags, Cabal notes
the directories in the package DB, and uses them when calling GCC.  It
doesn’t just use the directories specified when configuring a given
package.  It uses all the directories specified for all of that
package’s dependencies, too.

Right now, we pass in every “lib” and “include” directory from every
input, including the “standard-packages” from the GNU build system.
This means that Gawk’s “lib” directory (for example) is included every
time we configure any package.  In turn, when GHC calls GCC as part of
compiling a Haskell package, Gawk gets included on the command line once
for every node on that package’s dependency tree.  Packages with a large
dependency tree hit the 128K limit imposed by the kernel Linux.

What’s funny is that not a single Haskell package even uses shared
libraries from Gawk!

The other thing that’s funny is that most Haskell packages build just
fine without “extra-lib-dirs” and “extra-include-dirs”.  Even those that
call out to C code work, because Guix sets “LIBRARY_PATH” for us.
Unfortunately, packages that depend on packages that call out to C code
fail.  This is because Guix doesn’t set “LIBRARY_PATH” for the
transitive input, but GHC still wants to make use of the shared library.
This can be solved by adding back the “extra-lib-dirs” and
“extra-include-dirs” flags as-needed.

With this approach, I was able to compile Stack (the tests failed,
though).

The part that I’m still thinking through is how to make this whole
system work.  I would really like something that works automatically,
but I don’t see a way to do it.  This means we would have to add the
flags manually to the handful of packages that need them.  The part that
I really don’t like is that we don’t know a package needs the flags
until a package that depends on it fails to build.  Having everything
one step removed like that is not ideal.  On the other hand, there are
probably less than a dozen Haskell packages that need flags – it’s not
that common to mix Haskell and C code.  I will make a patch that does
this and see what the damages are.

This feels like a step in the right direction, because we end up with a
cleaner package graph as a result.  The added work and delayed feedback
will be tricky, but it should only come up infrequently.  What do you
think?


-- Tim