Re: "ERROR I/O" on a large database on version 0.7.9 Win7 binary

[John Darrington]

I had a look at those files.  I think perhaps there are several issues:

Firstly, the .SPS file uses DATA LIST FIXED, but the file you are 
trying to import does not have fixed data widths.  Importing a .csv file
like this is better done using GET DATA /TYPE=TXT

Secondly, although you specifid a numeric type for the first variable,
not all the data have numeric values in this field.  Line 22 for example
starts with V3001  You either need to change HOSPID to a string variable,
or change all the values to numeric ones.

Once I did these things your data imported fine.

The fact that you say there is a problem at arung 51000 lines however suggests
there may be another issue.  We occasionally get such reports from windows
users, but it doesn't seem to be a problem for other systems.

J'



On Fri, Jun 01, 2012 at 07:53:12PM +0000, Gong, Henry wrote:
     
     I would like to run a 2GB CSV database (3.4 million cases, 159 variables) 
on Win7.
     However,
     Somewhere between 38.2MB (51450 lines) and 38.2 MB (51475 lines), PSPP 
breaks down -- it can't import the dataset. I'm guessing that the number is 
based on something wrong with the windows binary or some temp access problem. 
Instead of the dataset being truncated like I've seen in other posts, though, 
every cell in the GUI is simply blank (or, in the command line, "ERROR I/O"). 
Running as administrator doesn't change this, and neither does obtaining user 
permission to access \Windows\Temp (which makes sense, since running as admin 
didn't work). Here's some system information:
     OS: Windows 7 Home Premium 64-bit
     HDD: ~200 free GB
     2 cores @ 3.4 GHz, 12GB RAM
     PSPP: psppire.exe 0.7.9 March 15, 64 bit from http://pspp.awardspace.com/, 
though I've tried probably 10 different versions and gotten the same result, 
including the most recent version and the other builder's version.
     
     
     When I run command line I get no error until LIST is run; then I get 
"Error I/O" and also an error writing to tmp (could not see exact syntax using 
LIST since the table began outputing thousands of lines of "ERROR I/O"). I get 
a large number of these in the syntax:
     C:\Users\User\file\path\to\csv\database.csv:87.183-87.187:warning: Data 
for variable VAR036 is not valid as format F: Field contents are not numeric.
     and one of these:
     note: Warnings (101) exceed limit (100). Syntax processing will be halted.
     
     The actual data is under a Data Use Agreement so I can't post it here, but 
here's the SPSS load program:
     
http://www.hcup-us.ahrq.gov/db/nation/kid/tools/pgms/SPSSLoad_KID_2009_Core.SPS
     
     I've tried making a non-sensitive information version of the data, and 
seem to have succeeded, in that I get the same results. The first ten variables 
are similar to DX1-10, and the rest of the variables are just there to bulk up 
the file size. Here it is:
     http://www.mediafire.com/?86rvxjf76vh53du
     
     

-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://keys.gnupg.net or any PGP keyserver for public key.

signature.asc
Description: Digital signature