Re: possible memory wasting in array handling

[Olaf Till]

On Tue, Jan 24, 2017 at 04:12:49PM -0500, John W. Eaton wrote:
> On 01/20/2017 04:15 AM, Olaf Till wrote:
> >On Thu, Jan 19, 2017 at 03:16:44PM +0100, Olaf Till wrote:
> >>A more difficult way would
> >>involve to maintain, in ArrayRep, not only a refcount, but also a
> >>notion of the greatest slice referenced by any Array.
> >
> >I'm more and more convinced that the problem caused by the described
> >behaviour is serious. Would you consider a patch?
> >
> >(I have a test case, too. It's machine dependent and on GNU/Linux
> >involves switching to single user mode and turning off swapping.)
> 
> It should be fairly easy to make the array slice optimization optional.
> 
> If we make change things so that Octave chooses slices vs. copying based on
> the slice to full array size, then that should probably also be configurable
> in some way.

Not yet using the patch tracker, since probably useless:

Testing the patch (single user mode, swapoff), unpatched gives:

octave:2> a = ones (500000000, 1);
octave:3> b = a(1:10);
octave:4> clear a
octave:5> a = ones (500000000, 1);
error: out of memory or dimension too large for Octave's index type
octave:5>

and in patched, as excepted, this can be continued further:

octave:1> a = ones (500000000, 1);
octave:2> b = a(1:10);
octave:3> clear a
octave:4> a = ones (500000000, 1);
octave:5> c = a(1:10);
octave:6> clear a
octave:7> a = ones (500000000, 1);
octave:8> ... and so on.

But if the original array is allocated in a function, even unpatched
has no problems:

octave:1> function ret = mkarray ()
> a = ones (500000000, 1);
> ret = a(1:10);
> endfunction
octave:2> b = mkarray;
octave:3> c = mkarray;
octave:4> d = mkarray;
octave:5> ... and so on.

Seems returning from function involves Array::make_unique(), which I
didn't realize.

Since the case with the function seems more typical than using 'clear'
in a script I'd say I'll dump my patch... (?)

Olaf

-- 
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc
Description: Digital signature