Re: OCTAVE_LOCAL_BUFFER

[Jaroslav Hajek]

On Sun, Dec 14, 2008 at 2:39 AM, John W. Eaton <address@hidden> wrote:
> On 12-Dec-2008, Jaroslav Hajek wrote:
>
> | This optimization is now uploaded. Octave will use a stack of 32MB
> | chunks and serve local buffers < 4MB for
> | the registered POD types (char, s/u short, int, long, float, double,
> | Complex, FloatComplex) from this stack.
>
> Thanks.
>
> Have you looked at how much memory is used just to get to the Octave
> prompt in typical usage?  Is it typical that multiple chunks will be
> alloated if we use 32MB?
>

I guess that depends on what typical usage is, but I guess that no. A
simple experiment I've just conducted shows that the whole "make
check" process ran on a single chunk, allocating ~12500 buffers in
total. Especially sparse matrix manipulations appear to use local
buffers extensively. A tribute to the coders - apparently they
anticipated the possible optimization. But I think that there are
still a couple more places where an Array or a std::vector or
something else is used as a local buffer.

Maybe we can experiment with the best settings here. For instance,
buffers larger than 1/8 chunk size will be allocated as stand-alone,
so that there are always at least 8 buffers per chunk. But it's
another guess - maybe 1/4 would suffice.

> | I think the 32MB size should at least be configurable, but I'm not
> | sure about the best way to do it. An environment variable, configure
> | switch, or sthing else?
>
> The simplest thing would probably be a function, but that would only
> apply to subsequent allocations, not the first chunk.
>

The code now assumes that the chunk size is constant. IMHO the optimal
chunk size mostly depends on the host machine's memory and also
typical data size the user is working with. I guess that the 32MB
default is fit for most normal computers; however, there should be a
way to make this configurable. Since there's no need to have this
fixed during compilation, what about an environment variable?


> jwe
>
>



-- 
RNDr. Jaroslav Hajek
computing expert
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz