Re: Building on OSX without package managers

[John W. Eaton]

On 24-Jan-2011, Richard Campbell wrote:

| I have 12 GB of RAM and I intend to use this on other machines with
| 16+ GB. Today I'm trying to load a contiguous block of 920 MB of
| complex doubles written to disk from some C code. Sometimes it
| works, sometimes it gives me an out of memory error (the kind where
| you have to restart Octave and try again).

Precisely how are you trying to load the data?  Is it written in a
binary or text format?  If text, you can greatly improve Octave's
performance for reading if you write a header like the one produced by
Octave's "save -text" command so that you can use load to read the
file and Octave won't have to guess the array size and possibly
reallocate repeatedly to read the data.

If you are reading a text file, is the 920MB the size of the text
file, or do you mean that you have that 920MB/16 ~= 115e6 complex
array elements?

If the array is 115e6 elements, that is not large enough to need 64-bit
indexing unless you generate much larger temporary values while you are
working with your data.  But with Octave (or Matlab), if you do
anything with the entire array, then you can expect to need
generate some temporaries that are the same size as the original
array.  But with current versions of Octave (3.3.x and later) the
limit on the total amount of data is much greater than 2GB if you have
a system with 64-bit pointers, even if you are building Octave to use
32-bit indexing (see the info in the manual about building with
--enable-64 for more details).

jwe