bug-bash
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

while read line; do; done; cannot handle big files


From: schomake
Subject: while read line; do; done; cannot handle big files
Date: Tue, 28 Apr 2009 19:50:07 +0200

Configuration Information [Automatically generated, do not change]:
Machine: i686
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='i686' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i686-redhat-linux-gnu' 
-DCONF_VENDOR='redhat' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib  -D_GNU_SOURCE 
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g -pipe -Wall 
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector 
--param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic 
-fasynchronous-unwind-tables
uname output: Linux ai17872 2.6.22.9-61.fc6PAE #1 SMP Thu Sep 27 18:27:50 EDT 
2007 i686 i686 i386 GNU/Linux
Machine Type: i686-redhat-linux-gnu

Bash Version: 3.1
Patch Level: 17
Release Status: release

Description:
        [Detailed description of the problem, suggestion, or complaint.]

The script

   while read line
   do
     echo "$line"
   done < larger-than-2GB-file.txt

will fail, returning erratic (shifted content) in $line after the 2GB has
been read.

Repeat-By:
        [Describe the sequence of events that causes the problem
        to occur.]
The input file is: big.dat
with 4357 white-space delimited items on each text record. 
The first item is a long (>80 bytes) alphanumeric ASCII string, the
remaining 4356 items are ASCII floating-point values. With exception
of the NEWLINEs (ASCII=10,dec) the file does not contain byte values
below decimal 32 or above 127, decimal.

wc big.dat
    115813  504597241 2528513389 big.dat
    lines   items     bytes


Fix:
        [Description of how to fix the problem.  If you don't know a
        fix for the problem, don't include this section.]

Wrote a program in C to check whether the file was corrupt.
It was not.




reply via email to

[Prev in Thread] Current Thread [Next in Thread]