Re : Bash scripting and large files: input with the read builtin from a redirection gives unexpected result with files larger than 2GB.

[Jean-François Gagné]

Hi Andreas and Chet,

@Andreas: this patch seams promising.  I am unable to test it now, but I will 
at the beginning of next week.

@Chet: according to the specification, the retrun value of lseek is -1 if 
error, or new offset from the beginning of the file if success.  In my case, 
the problem is that the call succeed, but r is negative due to the cast to int.

Thanks to you both,

Jean-François Gagné


________________________________
 De : Chet Ramey <address@hidden>
À : Andreas Schwab <address@hidden> 
Cc : "address@hidden" <address@hidden>; "address@hidden" <address@hidden>; 
Jean-François Gagné <address@hidden>; address@hidden 
Envoyé le : Dimanche 4 mars 2012 23h32
Objet : Re: Bash scripting and large files: input with the read builtin from a 
redirection gives unexpected result with files larger than 2GB.
 
On 3/4/12 3:51 PM, Andreas Schwab wrote:
> Bob Proulx <address@hidden> writes:
> 
>> Chet Ramey wrote:
>>> Jean-François Gagné wrote:
>>>> uname output: Linux xxxxxxxx 2.6.32-5-amd64 #1 SMP Tue Jun 14 09:42:28 UTC 
>>>> 2011 x86_64 GNU/Linux
>>>> Machine Type: x86_64-pc-linux-gnu
>>>
>>> Compile and run the attached program.  If it prints out `4', which it does
>>> on all of the Debian systems I've tried, file offsets are limited to 32
>>> bits, and accessing files greater than 2 GB is going to be unreliable.
>>
>> Apparently all of the Debian systems you have tried are 32-bits
>> systems.  On the reporter's 64-bit amd64 system it will print out 8.
> 
> But it won't help if you don't use it.
> 
> diff --git a/lib/sh/zread.c b/lib/sh/zread.c
> index 0fd1199..3731a41 100644
> --- a/lib/sh/zread.c
> +++ b/lib/sh/zread.c
> @@ -161,7 +161,7 @@ zsyncfd (fd)
>       int fd;
>  {
>    off_t off;
> -  int r;
> +  off_t r;
>  
>    off = lused - lind;
>    r = 0;

That's true, and I made this change some months ago.  The question is
whether or not it makes a real difference, since the only use of that
variable is to check whether the return value from lseek is -1.  I
suppose under the right set of circumstances it could.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
         ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRU    address@hidden   http://cnswww.cns.cwru.edu/~chet/