Re: [Bug-tar] Can't extract file from multivolume archive

[Alford, Seth]

On Mon, Feb 21, 2011 at 04:53:50PM -0600, Alford, Seth wrote (edited for 
spelling:)
> On Fri, Feb 04, 2011 at 06:43:09PM -0600, Paul Eggert wrote:
> > On 02/04/11 07:21, address@hidden wrote:
> > > Any ideas?
> > 
> > Is there some simple, self-contained shell script
> > that reproduces the problem without requiring access
> > to special hardware?  That might help us diagnose it.
> > At least we could add it to the test cases.
> > 
> > Otherwise, I dunno, I suppose you can use GDB to figure
> > out what went wrong.
> > 
> 
> I saw the same problem which stinga described, twice.  
> 
> Once, like stinga, in a non-portable environment.  
> 
> Once, while I was writing a script, test_multiple_volumes, where
> I tried to get the problem to happen again.
> test_multiple_volumes is attached.  
> 
> I have not been able to get the problem to repeat.
> 
> So I think that the problem which stinga saw exists, but it is
> intermittent.
> ...

I suspected that the problem might only occur when the backups
and files were larger.  So, I rewrote test_multiple_volumes to
take parameters.  Among the parameters it accepts are the number
of loopback disks, the size of the loopback disks, the size of
the files to back up, the files that the script should try to
delete, whether the script should re-run the test, and whether
the script should re-create the test files for the 2nd and
subsequent runs.

And, of course, I can't get the problem to repeat.  I can restore
files from individual tar volumes of a multi-volume tar, just
fine.  I plan to make this post to the list, and go back to doing 
what I was doing before I saw the problem.

The caveats I included with the first revision, concerning not
running this on an important system with important data, still
apply.

The updated script is attached, below.  Here's the output from
the usage function, which explains how to call the script.

---usage function output starts----
Usage:

./test_multiple_volues [ -n ] [ -c ] [ -d loopback_disk_size ]
        [ -D number_of_loopback_disks ] [-t testfile_size ]
        [ -l comma_separated_list_of_files_to_delete_and_restore ]
        [ -m ] [ -f ] [ -v ] 

where
-c makes the script run continuously (default: runs once)
-d specifies the loopback disk size.  Accepts "dd-style" numbers.
   (default: 100000000)
-D specifies the number of loopback disks (default: 3)
-f create loopback devices once at the start of the script with
   dd and mke2fs, then re-use them for each loop (default:
   recreate loopback devices during each time the script re-runs
   itself.)
-F create the test files during the first loop, then re-use them
   each time.  The test files are not initially stored on the
   loopback devices, so -F does NOT imply -m or -f.
-h outputs this help message and exits
-l specifies a list of file numbers to try restoring (default:
   36,41,42,47,51,60,64,71,80)  These get translated to the
   full testfile name, ie, testfile00036 .
-m leave loopback devices mounted for the duration of the test
   (default: umount each disk after its read or written)
   The -m flag implies -f.
-n makes the script non-interactive (default: interactive)
-t specifies the size of the files which fill the disks.  Accepts
   "dd-style" numbers (default: 3000000)
-v outputs the version number and exits

By default, the script creates 3 loopback disks which are each 100
Mbyte big.  The script creates a enough files to fill the loopback
disks to 90% overall, which with the default sizes is 90 files.
The script writes the files to the loopback disks using a multi-volume
tar command.  Then the script then deletes the files specified with
-l (or the default list if no -l was specified) and tries to restore
those files from the archives, using both a multi-volume tar and
a tar which reads from individual volumes.

With -m, the script will leave the script will create the
loopback disk once, then leave it mounted on the system.

With -f, the script assumes that loopback disks already exist
on the system.  The script does not attempt to (re)create them.

The -d and -t options take the same size extensions that dd does.
That is, 3M for 3 * 2 ^ 10.  See the dd man page for details.

---usage function output ends----

If you do experiment with this, be aware also that the script
leaves it to you to decide what backup members occur wholly on
one of the individual multi-volume archives.  What I have been
doing is watch it once, see which testfiles went onto what
backup, and then let it fail because the default testfiles are
incorrect.  Then I re-ran the script, supplying a -l parameter
with a list of testfile numbers that I observed did not span a
volume boundary.  For example, I've got the updated script
running on a test system with the parameters:

./test_multiple_volumes -c -n -F -m -d 1G -t 80M -D 5 -l 
13,17,18,27,29,33,41,45,48,50,55

Again, see the usage function for an explanation of what these
arguments do.

Yes, I know, the script should parse the output from tar when it
makes the backup, so it knows which backup members span a volume
boundary.  Then, during a restore testing, the script should
randomly choose among backup members that do not span volume
boundaries.  And the script would be easier to modify, and
probably shorter, if it were written in a scripting language such
as Python.  Had I known it would get this involved, I would have
written it in Python.  But it was just supposed to be a quick
script I wrote on a Sunday afternoon.

stinga, and anyone else, if you see the problem again, please tell us if it
is just --extract that fails.  When I first saw the problem, I thought that
I could see a backup member using --list, but then I could not get it out
using --extract.  Now that I think about it some more, that may mean that
there is some issue with how pathnames are interpreted in a multi-volume
archive.

Thanks,
--Seth Alford
address@hidden


This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.

test_multiple_volumes
Description: Text document