RE: [Bug-tar] GNU Tar 1.16.1 size limitation on multiple tape arc hive

[Petcher, Daniel]

Great suggestions, Joerg!

I'd love to turn my LTO-3 drive's HW compression off, but I'm not sure how.
I'll address that issue later.

I don't have the option of changing my data to refrain from including *.tar
files on my tape; others have stored these data as their own archive on the
server, and I'm dumping their archives (among other data) to tape with tar.

I don't think merely having a *.tar.bz2 is a problem. There are 128 other
*.tar.bz2 files, 102 *.tar.gz, and 1643 *.zip (mostly from WinZip) files
that verified before I reached the 10 GB *.tar.bz2 file that stopped me.

I've done some testing with a smaller data set that still includes my 10 GB
problem file so that I can keep the task entirely on a single tape. When I
tar-ed just the parent folder of this huge file, I still have the error, so
it's probably not a physical EOT detection issue.

I just tar-ed a folder that has even LARGER files that aren't tar archives
to try and rule-out the problem of misinterpreting the *.tar data. Bigger
files still fail: I've confirmed that my tar --diff loses sanity when it
tries to verify a 77 GB binary file.

Let's see what happens if I build a tape archive with a few 2 GB files
followed by the 77 GB file and the 11 GB *.tar.bz2 file. I'll try it with
and without the --multi-volume or --new-volume-script parameters to see if
they have an effect....

Either way, tar loses sanity while trying to verify a 77 GB file:
        address@hidden:/tmp# cat tard.out
        V--------- 0/0               0 2007-12-14 20:07 --Volume Header--
        drwxr-xr-x root/root         0 2007-12-11 12:11
home/departed_employees/BOBW/Laptop/
        -rw-r--r-- root/root 2147468973 2007-08-20 09:04
home/departed_employees/BOBW/Laptop/07082001.GHS
        -rw-r--r-- root/root 2147473946 2007-08-20 09:17
home/departed_employees/BOBW/Laptop/07082002.GHS
        -rw-r--r-- root/root 2147474209 2007-08-20 10:41
home/departed_employees/BOBW/Laptop/070820.GHO
        drwxr-xr-x root/root          0 2007-12-11 02:43
home/departed_employees/BOBW/Desk/
        drwxr-xr-x root/root          0 2007-12-11 10:32
home/departed_employees/BOBW/Desk/LATEST/
        -rw-r--r-- root/root 83052990464 2007-07-19 00:10
home/departed_employees/BOBW/Desk/LATEST/070718.tib
        ?rwsrwsrwt -1/-1 18446744073709551615 1969-12-31 15:59
)G\034b(\017»4ÛDxÇ unknown file type ` '
        tar: Archive contains address@hidden' where numeric off_t
value expected
        tar: Archive contains `\027À\200 ZÇý\036' where numeric mode_t value
expected
        tar: Archive contains `ÀÝô\216.À\2364Ø\231 æ' where numeric time_t
value expected
        tar: Archive contains `Ô\002Üã( íC' where numeric uid_t value
expected
        tar: Archive contains `(\223ê (\201Ah' where numeric gid_t value
expected
        tar: Archive contains `Ô\002Üã( íC' where numeric uid_t value
expected
        tar: Archive contains `(\223ê (\201Ah' where numeric gid_t value
expected
        tar: )G\034b(\017»4ÛDxÇ: Unknown file type ` ', diffed as normal
file
        tar: )G\034b(\017»4ÛDxÇ: Warning: Cannot stat: No such file or
directory
        tar: Skipping to next header


Is there some command-line option I can use to overcome this problem, or can
I recompile tar with some different options? I would prefer to use tar for
the sake of its nearly universal compatibility. If I can't get tar to work,
I'll try star. If that doesn't work for me, either, I'll have to resort to
commercial software. <GROAN>

Vielen dank!

-dP


address@hidden wrote on Friday, December 14, 2007 3:27
AM:

>Writing data compressed by a tar implementation to a real tape drive is
always a bad idea.
>
>Is GNU tar unable to deal with EOT detection while writing to a HW
compressed tape?


>"Petcher, Daniel" <address@hidden> wrote:
>
>> I'm running SlackWare 12 with GNU tar 1.16.1 to throw a huge pile of 
>> servers' backup data onto multiple LTO-3 tapes. The --create phase 
>> works well, but when I rewind tapes and --diff the tape archive, the 
>> process always stops on a certain 10 GB *.tar.bz2 file. I sorted my 
>> logs by filesize and I see that there are no previous files larger than 3
GB.