|
From: | Mariano Wis |
Subject: | Reading big files without Memory exhausted message? |
Date: | Wed, 14 Apr 2004 00:41:51 +0200 |
Hi all,
The summary of this message is:
Can Octave be
configured in any way to read big files and write them in big
arrays without the "memory exhaust -- trying to return propmpt" message and
the fail of the program?
I'm trying to read a big file
that is organized in samples of 7 doubles each. I have used two different
strategies to read this file (see sample codes below). When I try to read a
file that is not too big (about 150 Mbytes) I have no problem to read it.
The problems arises when I try to read bigger files (for example 230
Mbytes). At a certain point, I began to read the message: "memory exhaust --
trying to return propmpt" and the program window is closed. This happens at
differents points of the sample codes. With sample code 1 the message is shown 2
times after the last fread. With sample code 2 the message is shown 7 times
after the first fread. In both cases the Octave program is aborted and the
window is closed.
I'm
using octave-forge-windows 2.1.50 windows installation on a 1 Gb RAM /
Athlon XP 2400+ system running Windows XP and on a 512 Mb RAM /
Pentium4 system running Windows 2000 and I have the same effects on both
systems.
My questions are:
- Is the problem related with the
configuration of Octave/Cygwin?
- If so, can this configuration be changed to be
able to read and store in memory such big files?
Thanks beforehand,
Mariano Wis
## *** SAMPLE CODE 1 ***
imu_file =
'c:\\tstsds\\input_files\\imu_20030701.dat';
n_fields = 7; ## Number of fields for each
acquisition
data_size = 8; ## Size of each data (8 is double) f_info =
stat(imu_file);
f_size = f_info.size / (n_fields * data_size); ## Number of samples f_id = fopen(imu_file,'rb','native');
time = fread
(f_id,f_size,'double',48,'native');
fseek (f_id,8,SEEK_SET); ## Locate to read wx
wx = fread (f_id,f_size,'double',48,'native'); fseek (f_id,16,SEEK_SET); ## Locate to read wy
wy = fread (f_id,f_size,'double',48,'native'); fseek (f_id,24,SEEK_SET); ## Locate to read wz
wz = fread (f_id,f_size,'double',48,'native'); fseek (f_id,32,SEEK_SET); ## Locate to read ax
ax = fread (f_id,f_size,'double',48,'native'); fseek (f_id,40,SEEK_SET); ## Locate to read ay
ay = fread (f_id,f_size,'double',48,'native'); fseek (f_id,48,SEEK_SET); ## Locate to read az
az = fread (f_id,f_size,'double',48,'native'); fclose(f_id);
## *** SAMPLE CODE 2 ***
imu_file =
'c:\\tstsds\\input_files\\imu_20030701.dat';
n_fields = 7; ## Number of fields for each
capture
data_size = 8; ## Size of each data f_info = stat(imu_file);
f_size = f_info.size / (n_fields * data_size); ## Number of samples f_id = fopen(imu_file,'rb','native');
imu_vec = fread (f_id,[n_fields,f_size],'double',0,'native'); time = imu_vec(1,1:f_size);
wx = imu_vec(2,1:f_size); wy = imu_vec(3,1:f_size); wz = imu_vec(4,1:f_size); ax = imu_vec(5,1:f_size); ay = imu_vec(6,1:f_size); az = imu_vec(7,1:f_size); clear imu_vec;
fclose(f_id);
--- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.655 / Virus Database: 420 - Release Date: 09.04.2004 |
[Prev in Thread] | Current Thread | [Next in Thread] |