[Octave-bug-tracker] [bug #42118] COW memory issues when extracting small slices from large arrays

[Lachlan Andrew]

Follow-up Comment #5, bug #42118 (project octave):

This was put on hold for 4.0.0, but with a bugfix 4.0.1 coming up, should it
be revisited?

Michael, you suggested cleaning memory if the reference count drops to 1. 
What if there are two small slices?  The object would still stay in memory.

It may be better to have a separate slice counter, or even a list of slice
objects, associated with array data types.  If the slice counter equals the
the object counter, then you could do whatever memory optimisation.  The
existence of this extra data shouldn't slow down any non-slice operations
except for deletion (except for the slightly reduced memory locality).

One problem with jwe's idea of copying if the fraction is small, is that it is
common to walk through a matrix row at a time or column at a time.  These
slices are likely to be small fractions of the total size (1% or 0.1%).  It
would be a shame to lose all copy-on-write benefits for that common case.


This is a bug example of a problem I've often thought about: whos shows the
nominal memory sizes of objects, but doesn't show how much overhead there is. 
For example, a huge sparse array can have zero bytes of data, but still have a
huge amount of column overhead.  It would be great to have a version of whos
that shows memory *consumed* by each variable.  Is that feasible?

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?42118>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/