Re: [Bug-tar] GNU Tar 1.16.1 size limitation on multiple tape arc hive

[Joerg Schilling]

"Dat Head" <address@hidden> wrote:

> > "Petcher, Daniel" <address@hidden> wrote:
> > > I'd love to turn my LTO-3 drive's HW compression off, but I'm not sure 
> > > how.
> > > I'll address that issue later.
>
> mt -f /dev/stX defcompression off
> should work - works on all the other drives I've used over the years
> (4mm, 8mm, DLT, SDLT I, SDLT II, etc)

Compression is controlled via SCSI density codes and density is selected via
different tape node entries.

There are different letters (l m h c) Low, medium, high, compression.

This is how tapes interfaces work since more than 20 years. What platform are 
you using?


> On Dec 15, 2007 1:20 PM, Joerg Schilling
> <address@hidden> wrote:
>
> > I believe that this is an internal GNU tar problem and this is why I 
> > suggested
> > to try out star.
>
> gnu tar --listed-incremental still does not work properly at all if
> you have millions of files (8.5 on
> the one box where it failes) and
> list several different file systems (about 8 on the one example
> system) on the command line - it just *silently* does not back up
> any files in some of them - if you have either of these situations (#
> of files and multiple
> file systems I highly suggest you check your output, the -v log file
> is sufficient, as it just
> does not list anything for them)

Implementing incremental backup/restore on more than filesystem at a time
is something that adds a lot of complexity to backup/rerstore. This is 
why programs that like a working implementation do not support it (at least in 
the first attempt).

I see no reason why backups cannot be split down to the underlying filesystems
by the user who sets up the backup program.


> sadly it does not appear that
> this will ever be fixed in gnu tar which I do not understand as I have
> been a hugh champion of
> gtar for so long but can no longer risk my data to it, so is there any

GNU tar did introduce this feature in 1992 and from my tests, it did never work.
I cannot speak anymore for older versions as I don't remember the results.
The last test I run 3 years ago was unable to restore backups correctly that
include renamed directories.

>From my understanding, the problem with GNU tar is that it does not archive
enough meta data information _inside_ the backup but instead needs a separate
file between backups that stores inode numbers for directories. Star uses a 
different method - the same basic method as used by ufsdump/ufsrestore since
more than 25 years. Star just remembers the filesystem, the "dump level" and
the time stamp of the related backup start. Star simply archives all files
that have been touched since the last backup of a higher level than the current.
As star puts all file meta data into the archive, the restore does all the other
work by creating a data base that includes file path names together with the 
"original inode number" and the "restored inode number". UNIX backup systems 
before UFS (before 1981) did work the same way with backups but restored on
_unmounted_ filesystems. This alloed them to use the same inode number for the
restore.

This method of incremental thus has a > 30 year period of testing. There is 
another system that may work. It has been introduced by David Korn and Glenn 
Fowler and is called "differential backup" because it is based on file 
differences and checksums on file parts.


> cheat sheets on how to convert over
> to star as far as incremental backup schemes go (or alternatively just
> what would be missed by
> using "find | gtar" [what flexbackup uses] vs. "gtar --listed-incremental" ?)

What GNU tar does is not really "incremental backups", this is why I call
star's method "true incremental backup".

If you like to use find, you could better use star's build in find as this
allows you to do more than you could with a separate call to find. You still
get something that is different to a true incremental backup. 

A true incremental backup allows you do incremental restores that honor renamed 
files/directories and removed files/directories. If you just do a time stamp 
based backup (as you could do with find), you are able to retrieve single
files from the backup without problems but you cannot simply restore the whole
filesystem from the last full dump and the incrementals.

Jörg

-- 
 EMail:address@hidden (home) Jörg Schilling D-13353 Berlin
       address@hidden                (uni)  
       address@hidden     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily