Re: [PATCH] build: pull: Compile .scm files in one process.

[Andy Wingo]

On Fri 06 Nov 2015 16:41, address@hidden (Taylan Ulrich "Bayırlı/Kammer") 
writes:

> Andy Wingo <address@hidden> writes:
>
>> On Thu 05 Nov 2015 17:10, address@hidden (Taylan Ulrich "Bayırlı/Kammer") 
>> writes:
>>
>>> It used to max out every CPU core, now just one. :-)
>>> It used to take ~18 minutes on my machine, now less than 3.
>>
>> If you compile within a par-for-each you should be able to peg your CPU
>> core again, but actually reduce the time :)
>
> From what I understand, that would probably ignite the bug again.
>
> We need to ensure that as soon as a module file is compiled, it's also
> explicitly loaded before anything else is compiled (which might import
> it), otherwise that compilation will import the "degenerate" version of
> the module that results from compiling but not loading it.
>
> But that's really just my shallow high-level understanding of the bug,
> and could be way off.  If you have any insights on what's really going
> on, that would be greatly appreciated. :-)

AFAIU the problem that makes compilation slow is that *expansion* is
slow.  When all your Scheme files are fresh, compiling 1 module has to
expand all N modules.  Using one process to compile avoids this N^2
penalty, just paying the O(N) cost up-front and then the marginal
compilation cost is O(1).

There is benefit to compiling support modules before compiling (gnu
packages) so that Guix's macros run compiled and not interpreted, but if
you already have all of the modules expanded and loaded I don't think
there is any advantage to loading the freshly compiled .go files.

I do not understand what you mean by "degenerate" modules :)  An
interpreted module should act the same as a compiled module.  I am
interested to hear what difference you can perceive between the two.

Andy