[Bug-tar] listed-incremental confusion

[Paul Raines]

I have a RH7.3 linux box with tar-1.13.25

I have backup volume frigg/9 with a history like this:

DATE      TYPE  TAPE:FNUM  SIZE
05-02-27  FULL  AAH747:01  58871MB
05-03-10  INCR  AAH028:01  57345MB
05-03-20  INCR  AAH029:36  18147MB
05-03-30  INCR  AAH031:07  13510MB
05-04-09  INCR  AAH033:07  10962MB
05-04-21  INCR  AAH035:14   8796MB

Each backup was made like this:

  tar cvvlSf - -g /backup/volume/frigg/9/incr.dat --totals -C / \
        local/frigg/9 2> /backup/tape/$TAPE/$FNUM/list | \
        dd of=/dev/nst0 bs=64k >> /backup/tape/$TAPE/$FNUM/log 2>&1
  cp /backup/volume/frigg/9/incr.dat /backup/tape/$TAPE/$FNUM/incr.dat

where any existing /backup/volume/frigg/9/incr.dat was deleted
before the first FULL backup.

Note I have a listing of every file backed up at each run then in
/backup/tape/$TAPE/$FNUM/list.  I have short perl script that can
go through these listings to tell me for a given subdirectory and
date what tapes I need to restore.

===========================================
#!/usr/bin/perl

my $dir = 'local/frigg/9/users/kk/filter/lp20_jas_eq';
foreach my $file (@ARGV) {
  my($tapef) = ($file  =~ m!(AAH\d+:\d+)!);
  open(LIST,"<$file") or die "$!\n";
  while (<LIST>) {
    next unless /^-/; # consider only files
    my($size,$path) = m!(\d+) 200[ 0-9:-]+ $dir/(.*)$!o;
    next unless $path;
    $db{$path} = $tapef;
    $dbsize{$path} = $size;
  }
}

my $tot=0;
my %tapecnt;
foreach my $path (keys %db) {
  $tapecnt{$db{$path}}++;
  $tot += $dbsize{$path};
}
foreach my $tapef (sort keys %tapecnt) {
  print "$tapef: " . $tapecnt{$tapef} . "\n";
}
print "TOTAL: $tot\n";
============================================

The user wants a restore up to the April 9th backup date of her 
local/frigg/9/users/kk/filter/lp20_jas_eq directory.  So I feed into the 
script above the $TAPE/$FNUM/list files in order up to the 05-04-09 list
and I get:

AAH028/1: 269
AAH029/36: 263
AAH031/7: 5469
AAH033/7: 2158
TOTAL: 24574602397

So I think I should expect about at MAX 8159 files totalling 25GB,
but probably less due to deletions that happened between increments.
Also note the directory needed did not exist at the first FULL backup
so its archive is not needed.

I go to an empty volume and create a restore directory

  mkdir restore; cd restore
  cp /backup/tape/AAH033/7/incr.dat /tmp/incr.dat
  DIR=local/frigg/9/users/kk/filter/lp20_jas_eq

  loadtape AAH028 1
  dd if=/dev/nst0 bs=64k | tar xf - -g /tmp/incr.dat $DIR
  unloadtape AAH028
  loadtape AAH029 36
  dd if=/dev/nst0 bs=64k | tar xf - -g /tmp/incr.dat $DIR
  unloadtape AAH029
  loadtape AAH031 7
  dd if=/dev/nst0 bs=64k | tar xf - -g /tmp/incr.dat $DIR
  unloadtape AAH031
  loadtape AAH033 7
  dd if=/dev/nst0 bs=64k | tar xf - -g /tmp/incr.dat $DIR
  unloadtape AAH033

And I end up with only 756 files extracted.  In another shell
while the above was taking place I had counted the files before
the tape AAH033 extraction and got 756 so on the last tape,
no files at all were extracted even though there were clearly
2158 files that should be extracted!

At this point, I thought maybe listed-incremental works only
if you are extracting the whole archive and breaks if you give
file or directory selectors.  So I did the whole thing above
again, creating a blank new restore volume to extract to, but
this time leaving out DIR.  This time not a single file was
extracted.  Not a one.

Can someone explain how --listed-incremental is "really" supposed
to work and why it does not work in the above case?

--
---------------------------------------------------------------
Paul Raines                   email: address@hidden
MGH/MIT/HMS Athinoula A. Martinos Center for Biomedical Imaging
149 (2301) 13th Street        Charlestown, MA 02129     USA