RE: [Axiom-developer] --patch46

[Bill Page]

Camm,

On October 31, 2005 9:31 AM you wrote:
> 
> Greetings!  And please excuse me -- this thread has slipped under
> the radar here.

No problem.

> 
> Bill, here is the problem -- Redhat 9 is apparently placing the
> stack within the first 1Gb of memory, but you are requesting this
> amount for your heap.  GCL won't (or tries not to) allow the head
> to overrun the stack: 
> ... 
> In all of the machines I've looked at, this is about the worst
> stack placement I've seen.  Is this 'enterprise' or 'normal'?

Actually, I don't know very much about this server. It is a
"Unmanaged Virtual Servers (VPS/VDS)" hosted at a commercial
ISP called RoseHosting. (This is the system which runs MathAction
and for which Tim Daly is still paying the full operational
costs.) http://www.rosehosting.com

Can you deduce anything from the following system information:

  # uname -a
    Linux axiom-developer.org 2.6.11.10-RH1956 #9 SMP
      Fri May 20 20:48:44 CDT 2005 i686 i686 i386 GNU/Linux

I think that what we have is a legacy RedHat 9 configuration.
According to the RoseHosting website, we could probably upgrade
to a Fedora Core 4 configuration.

Perhaps if I contacted the admins at RoseHosting, I might be
able to get more details.

> I thought the former had a sophisticated mechanism to push
> the stack and the shared memory area as high up in memory
> as possible.
> 
> I know of no way to change this short of getting a different
> Linux kernel, where the issue should be addressable.
> 

I think the above info suggests we are running 2.6.11.10.
That's fairly new isn't it?

> One thing we could do is relax the power of two constraint
> on maxpages if a lesser amount would suffice.
>

Yes, very likely a lesser amount would suffice.
 
> 
> You say there is another work around with
> si::allocate-contiguous-pages, which I suspect to be a
> feather-edge manipulation issue, but am not sure.  You
> can check this out by doing (room) after your job has
> completed and seeing how close you are to OOM.

Ok. Here is the output of (room) in the case of setting
si::allocate-contiguous-pages, see:

http://wiki.axiom-developer.org/SandBoxCategoryOfGraphs

)lisp (room)
  1528/1528   59.3%       4 CONS RATIO LONG-FLOAT COMPLEX STRUCTURE
    94/200    99.9%         FIXNUM SHORT-FLOAT CHARACTER
                            RANDOM-STATE READTABLE SPICE
   340/500    75.9%         SYMBOL STREAM
     1/8      23.1%         PACKAGE
    65/400    99.0%         ARRAY HASH-TABLE VECTOR BIT-VECTOR
                            PATHNAME CCLOSURE CLOSURE
   500/500    53.3%       1 STRING
   195/331    99.8%       2 CFUN BIGNUM
    40/115    97.6%         SFUN GFUN VFUN AFUN CFDATA
  3000/3000               2 contiguous (237 blocks)
       1476                 hole
       1000   73.5%       1 relocatable
      2763 pages for cells
      8239 total pages
    116036 pages available
      6797 pages in heap but not gc'd + pages needed for
           gc marking
    131072 maximum pages
Value = NIL

-------

>From this output how can I tell how "close I am to OOM"? Maybe
the FIXNUM ... memory is a problem? What can I do about that?

> In addition to the contiguous page function you've found, here
> are some other ways to explicitly allocate memory to various
> types:
> 
> (si::allocate 'cons 400)
> (si::allocate-relocatable-pages 10000)
> <will explain the allocate sgc functions later if desired>

How can I related these to the output of (room)?

> 
> The typed pages are self-explanatory I'd think.  Contiguous
> pages are used for untyped memory which cannot be moved,
> typically compiled-code.  relocatable-pages hold untyped
> memory to which no pointer refers directly (i.e. outside of
> a lisp object, where we can control the reference in the
> garbage collector), string bodies, bignum bodies, certain
> arrays, etc.

Where do FIXNUM ... things live?

> 
> Here is a bit of clarification on the room report:
> 
>    211/211    42.9%   CONS RATIO LONG-FLOAT COMPLEX STRUCTURE
>    
>    ^  pages allocated for this type
>         ^  maximum pages before triggering GC
>               ^  Percent of the allocated pages holding live
>                  objects
> 
> =============================================================
> >(room)(si::allocate 'cons 400)(room)
> 
>    211/211    42.9%         CONS RATIO LONG-FLOAT COMPLEX
>                             STRUCTURE
>      2/28     40.8%         FIXNUM SHORT-FLOAT CHARACTER 
>                             RANDOM-STATE READTABLE SPICE
>     47/49     72.9%         SYMBOL STREAM
>      1/2      12.8%         PACKAGE
>      1/38     56.8%         ARRAY HASH-TABLE VECTOR
>                             BIT-VECTOR PATHNAME CCLOSURE
>                             CLOSURE
>     16/32     85.8%         STRING
>      3/27     98.3%         CFUN BIGNUM
>      6/115    88.1%         SFUN GFUN VFUN AFUN CFDATA
> 
>    347/512                  contiguous (86 blocks)
>        13107                hole
>        5242    0.0%         relocatable
> 
>        287 pages for cells
>      18983 total pages
>     101769 pages available
>      10320 pages in heap but not gc'd + pages needed for gc marking
>     131072 maximum pages
> 

Thanks for the help.

Regards,
Bill Page.