Re: [Gcl-devel] GCL on Mac

[Camm Maguire]

Greetings!

Formal documentation for these variables will appear in the info files
on release.  I'm still hoping to automate some of them away from the
user.

GCL_MEM_MULTIPLE multiplies the amount of apparent available memory on
startup to yield an effective runtime maximum.  So when doing
GCL_MEM_MULTIPLE=0.125 ... make -j 8, one is giving each job one eighth
of the runtime memory.

One can see how this might not be optimal, as certain jobs are likely to
require much more memory than others.  So GCL_MULTIPROCESS_MEMORY_POOL=t
... make -j 8 will keep a file in /tmp with the total GCL memory in use
across all jobs, and make allocation decisions based on this statistic.
I've experimented with this fairly extensively on seafoam, and while
basically stable, I have not yet found significant performance
improvements. 

GCL will not gc at all until GCL_PAGE_MIN fraction of the effective
runtime maximum number of pages is allocated.  So lowering this value
starts gc earlier.

Once started, GC will be triggered based upon the amount of allocation
since the last gc.  The threshhold is lowered smoothly as the heap
approaches GCL_PAGE_MAX fraction of the effective runtime maximum, at
which point GC will occur whenever the last allocation is GCL_ALLOC_MIN
fraction of the effective runtime maximum.

So for example, with the default settings, no gc occurs until 0.5 of
the runtime maximum memory is allocated.  Then GC starts whenever the
recent allocation is 1/3+0.05=38% of the runtime maximum, until the heap
grows to 0.75 of the maximum, at which time gc occurs whenever the
recent allocation is 5% of the maximum.

If any of this is unclear, please feel free to ask followup questions.
Suggestions for improvement as always are most appreciated.

Take care,

Matt Kaufmann <address@hidden> writes:

> Thanks, Camm.  Is there user-level documentation available for
> si::*code-block-reserve*, GCL_MEM_MULTIPLE, and
> GCL_MULTIPROCESS_MEMORY_POOL?  I found
> https://lists.gnu.org/archive/html/gcl-devel/2015-11/msg00018.html
> but, for example, it doesn't really tell me, as a dumb user, what to
> do.  For now I'll just use your suggestion below of
> GCL_MEM_MULTIPLE=0.125 for my -j 8 runs.
>
> Thanks --
> -- Matt
>> From: Camm Maguire <address@hidden>
>> Date: Sat, 23 Jan 2016 08:40:37 -0500
>> 
>> Greetings, and thanks so much for your feedback!
>> 
>> As you know one of the release goals of 2.6.13 is to be able to use all
>> available memory effectively, i.e. no compile-time maxpages.  This
>> introduced two problems, 1) loading code over 2Gb limit, and 2) staying
>> out of swap.  As for the first, we have si::*code-block-reserve*.  As
>> for the second, we have those tuning environment variables I've
>> mentioned before.  You should be good at present for example by setting
>> GCL_MEM_MULTIPLE=0.125, among other possibilities
>> (e.g. GCL_MULTIPROCESS_MEMORY_POOL).
>> 
>> Though this does unlock performance gains, I'm unhappy about the extra
>> tuning variables.  In general, I'd like things to 'just work' by default
>> and possibly have tuning variables available for optional use by
>> experts.
>> 
>> It is conceivable that I could have gcl recognize the 2Gb limit and
>> compile code with the slower large memory model once reached, though
>> this code will lose about 10%.  It is also possible that GCL does some
>> sort of load monitoring, somewhat like the MULTIPROCESS_POOL but from a
>> system-wide level, instead of gauging memory at startup.  The general
>> problem here is that GCL cannot shrink its heap, so the maneuverability
>> once near saturation is detected is limited.  There is a branch which
>> implements a copying collector that would alleviate this, but I am
>> hoping to avoid this for this release.
>> 
>> Anyway, suggestions as always most appreciated.
>> 
>> Take care,
>> 
>> Matt Kaufmann <address@hidden> writes:
>> 
>> > Hi, Camm --
>> >
>> > FYI, I tried the latest ACL2 and GCL ANSI on my Mac.
>> > Things went pretty well for awhile -- no failures -- but
>> > eventually memory usage got out of hand (I can send
>> > you screenshots of windows about that if you want) and
>> > I rebooted my machine.  Perhaps if I'd run make with
>> > something less than -j 8 I wouldn't have had this problem
>> > -- I saw at least one job of over 7 GB and I only have
>> > 16 GB of RAM.
>> >
>> > I rarely run GCL on my Mac anyhow, so no big deal;
>> > but I thought I'd mention this in case you're interested.
>> >
>> > -- Matt
>> >
>> >
>> >
>> >
>> 
>> -- 
>> Camm Maguire                                     address@hidden
>> ==========================================================================
>> "The earth is but one country, and mankind its citizens."  --  Baha'u'llah
>> 
>> _______________________________________________
>> Gcl-devel mailing list
>> address@hidden
>> https://lists.gnu.org/mailman/listinfo/gcl-devel
>> 
>
>
>
>

-- 
Camm Maguire                                        address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah