[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH] build: Speed up .go compilation.
From: |
Ludovic Courtès |
Subject: |
Re: [PATCH] build: Speed up .go compilation. |
Date: |
Thu, 14 Jan 2016 15:02:35 +0100 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) |
address@hidden (Taylan Ulrich "Bayırlı/Kammer") skribis:
> address@hidden (Ludovic Courtès) writes:
>
>> Mark H Weaver <address@hidden> skribis:
>>
>>> I haven't yet investigated, but my strong suspicion is that this is due
>>> to the fact that Guile's module system is not thread safe. More
>>> specifically, when a new module is created, it mutates the global
>>> directory of modules in a way that is not thread safe.
>>>
>>> New modules are created by 'compile-file', both for the module being
>>> compiled and for any imported modules that haven't been previously
>>> loaded. Unfortunately, this means that this approach of compiling files
>>> in multiple threads within a single guile process is not safe. There
>>> are likely to be random crashes and corruptions.
>>
>> Right. This is one of the concerns I raised before I forgot again. ;-)
>>
>> https://lists.gnu.org/archive/html/guix-devel/2015-11/msg00359.html
>>
>> Taylan writes:
>>
>>> The latest version of the patch loads all needed modules first, in the
>>> main thread, and then compiles the files corresponding to the modules in
>>> parallel. Does that still lead to mutation in the directory of modules
>>> in the parallelized segment of the code?
>>
>> It seems to be safe, but we’re treading in a risky zone.
>>
>> We’re pushing ‘compile-file’ and related code to its limits. It would
>> be great to address these issues in Guile itself.
>
> If we don't call compile-file in parallel but still do everything in one
> process, we will still be cutting down the time significantly.
>
> For instance, currently the load phase and compile phase take about one
> minute each on my machine with 4 cores. Making it non-parallel would
> mean the compile time takes 4x longer, i.e. 4 min., which brings us to 5
> minutes total. The original was around 20. On devices with fewer
> cores, the parallel compile is even less important.
>
> So in the worst case we can just turn that par-for-each into a for-each
> and still get benefits.
Right.
Mark, WDYT?
I would say: go for it, and let’s switch back to ‘for-each’ if/when we
have evidence of things going wrong.
I think that if things go wrong, that’ll be a hard failure (segfault or
some random run-time error) and not a silent hard-to-detect corruption.
Ludo’.
- Re: [PATCH] build: Speed up .go compilation., (continued)
- Re: [PATCH] build: Speed up .go compilation., Taylan Ulrich Bayırlı/Kammer, 2016/01/10
- Re: [PATCH] build: Speed up .go compilation., Mathieu Lirzin, 2016/01/10
- Re: [PATCH] build: Speed up .go compilation., Taylan Ulrich Bayırlı/Kammer, 2016/01/10
- Re: [PATCH] build: Speed up .go compilation., Mathieu Lirzin, 2016/01/10
- Re: [PATCH] build: Speed up .go compilation., Ludovic Courtès, 2016/01/11
- Re: [PATCH] build: Speed up .go compilation., Taylan Ulrich Bayırlı/Kammer, 2016/01/11
Re: [PATCH] build: Speed up .go compilation., Mark H Weaver, 2016/01/10
- Re: [PATCH] build: Speed up .go compilation., Taylan Ulrich Bayırlı/Kammer, 2016/01/10
- Re: [PATCH] build: Speed up .go compilation., Ludovic Courtès, 2016/01/11
- Re: [PATCH] build: Speed up .go compilation., Taylan Ulrich Bayırlı/Kammer, 2016/01/11
- Re: [PATCH] build: Speed up .go compilation.,
Ludovic Courtès <=
- Re: [PATCH] build: Speed up .go compilation., Ludovic Courtès, 2016/01/17
- Re: [PATCH] build: Speed up .go compilation., Taylan Ulrich Bayırlı/Kammer, 2016/01/18