Re: [Bug-tar] new/old extract problems with 6GB sparse file

[Joerg Delker]

Paul Eggert wrote:
> Joerg Delker <address@hidden> writes:
>
>   
> > So, I did some research on the list and found several issues with
> > equal or similar cases. However, they all (?) should have been fixed
> > in cvs already.
> >     
>
> Yes, that's correct.  However, some of the issues have to do with
> _creating_ tar files, not _extracting_ from them.  So, if you used a
> buggy version of tar to create the tar image, that image is corrupted,
> and you cannot extract from that image even if you use a non-buggy
> version.
>   

Is that because the extract logic is just not "aware" of this problem,
or because there was already data loss during creation?


>   
> > Any hints how to recover this archive is very welcome!    
>
> If the problem is what I think it is, you may be able to work around
> some of the problem by modifying the incorrect sparse-file offsets in
> the tar image.  However, in many cases the data will be lost, so I
> wouldn't get my hopes up.  (I haven't examined the code in detail, so
> I can't be more specific.)  For example:
>
>    $ echo '' | dd bs=1 seek=6292304383 of=jake.ntfs
>    1+0 records in
>    1+0 records out
>    1 byte (1 B) copied, 7e-05 seconds, 14.3 kB/s
>    $ ls -l jake.ntfs
>    -rw-r--r-- 1 eggert eggert 6292304384 Jun 19 02:22 jake.ntfs
>    $ tar-1.15.1 cSf buggy.tar jake.ntfs
>    $ tar-CVS cSf working.tar jake.ntfs
>    $ od -c buggy.tar >buggy.od
>    $ od -c working.tar >working.od
>    $ diff -u buggy.od working.od
>    --- buggy.od 2006-06-19 02:24:37.000000000 -0700
>    +++ working.od       2006-06-19 02:24:43.000000000 -0700
>    @@ -4,7 +4,7 @@
>     0000140  \0  \0  \0  \0   0   0   0   0   6   4   4  \0   0   0   0   1
>     0000160   7   5   0  \0   0   0   0   1   7   5   0  \0   0   0   0   0
>     0000200   0   0   0   1   0   0   0  \0   1   0   4   4   5   4   6   6
>    -0000220   3   3   1  \0   0   1   5   4   1   0  \0       S  \0  \0  \0
>    +0000220   3   3   1  \0   0   1   5   4   1   4  \0       S  \0  \0  \0
> This is just a checksum difference; not worth worrying about.
>
>     0000240  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
>     *
>     0000400  \0   u   s   t   a   r          \0   e   g   g   e   r   t  \0
>    @@ -12,11 +12,14 @@
>     0000440  \0  \0  \0  \0  \0  \0  \0  \0  \0   e   g   g   e   r   t  \0
>     0000460  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
>     *
>    -0000600  \0  \0   1   6   7   0   3   1   7   0   0   0   0  \0   0   0
>    +0000600  \0  \0   5   6   7   0   3   1   7   0   0   0   0  \0   0   0
> This is the bug in question: the correct number was 56703170000 (octal)
> but the leading "5" was incorrectly output as "1".
>
>     0000620   0   0   0   0   0   1   0   0   0  \0  \0  \0  \0  \0  \0  \0
>     0000640  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
>     *
>     0000740  \0  \0  \0   5   6   7   0   3   1   7   1   0   0   0  \0  \0
>     0000760  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
>     *
>    +0001760  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \n
>    +0002000  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
>    +*
> This is some data that went missing because of the bug.
>
>     0024000
>
> Obviously this is a very serious bug, which is why we're trying to
> generate a new official version soon.  

Thanks for the anaysis.
If I compare this to my archive, this doesn't match your comments:
0000600  \0  \0   0   0   0   0   0   0   0   0   0   0   0  \0   0   0
0000620   0   0   0   0   0   7   0   0   0  \0   0   0   0   0   0   0
0000640   2   0   0   0   0  \0   0   0   0   0   0   5   1   0   0   0
0000660   0  \0   0   0   0   0   0   5   6   0   0   0   0  \0   0   0
0000700   0   0   2   0   0   0   0   0   0  \0   0   0   1   0   3   4
0000720   2   0   0   0   0  \0   0   0   0   0   0   1   5   0   0   0
0000740   0  \0 001   5   6   7   0   3   1   7   1   0   0   0  \0  \0
0000760  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0

The length appears only at 0740 not 0600 and has a *leading* 001.

What about the additional data at 1760/2000. Is that static or does it
depend on the content?
Any best practices how to hex edit such large files?

Regards,
Joerg