RE: [Bug-tar] --listed-incremental still failing

[MrC]

> MrC ha escrit:
> 
> > After a change was made in the source directory, an incremental run 
> > should have picked up the changes.  But it fails to detect 
> any change at all.
> > Here's the ls difference for one of the modified files.  
> The changes 
> > were made on June 19th:
> > 
> > 
> > < -rw-r--r--    1 apache   apache     292638 2007-01-20 
> 16:32:56.000000000
> > -0800 photos.dat
> > ---
> > > -rw-r--r--    1 apache   apache     292638 2007-06-19 
> 17:06:16.000000000
> > -0700 photos.dat
> 
> Can you send me snapshot files created before and after these changes?
> 
> Regards,
> Sergey
> 

Sergey,

Thanks for you reply.  Up front, I need to apologize for my error, and that
my multi-level backup script is now re-written to address my oversight.
Thanks for your patience.

It was the exact wording of your request above for "snapshot files before
and after" that triggered sudden insight.

I had been using *one* .snar file per backup tree.  Although I'd read the
info page a dozen times, I kept missing a key point.  I was thinking the
documentation might be made to call out this important point, which I walk
through below:

>
>   To create an incremental backup, you would use
>`--listed-incremental' together with `--create' (*note create::).  For
>example:
>
>     $ tar --create \
>                --file=archive.1.tar \
>                --listed-incremental=/var/log/usr.snar \
>                /usr
>
>   This will create in `archive.1.tar' an incremental backup of the
>`/usr' file system, storing additional metadata in the file
>`/var/log/usr.snar'.  If this file does not exist, it will be created.
>The created archive will then be a "level 0 backup"; please see the
>next section for more on backup levels.

There is a subtle implication above that only 1 snar file is required.

>
>   Otherwise, if the file `/var/log/usr.snar' exists, it determines
>which files are modified.  In this case only these files will be stored

The "Otherwise" here also subtly implies that only a single snar file is
used.


>in the archive.  Suppose, for example, that after running the above
>command, you delete file `/usr/doc/old' and create directory
>`/usr/local/db' with the following contents:
>
>     $ ls /usr/local/db
>     /usr/local/db/data
>     /usr/local/db/index
>
>   Some time later you create another incremental backup.  You will
>then see:
>
>     $ tar --create \
>                --file=archive.2.tar \
>                --listed-incremental=/var/log/usr.snar \
>                /usr
>     tar: usr/local/db: Directory is new
>     usr/local/db/
>     usr/local/db/data
>     usr/local/db/index
>
>The created archive `archive.2.tar' will contain only these three
>members.  This archive is called a "level 1 backup".  Notice that
>`/var/log/usr.snar' will be updated with the new data, 

All the points above imply one snar file is required.  In fact, if one uses
just one snar file as shown above, the errors I described in my post with
empty tar archives, and then subsequent full backup archives, will occur.

There are some backup programs that create one, and only one, snapshot file
which is updated each run to reflect the state of the archives.  This is the
model I had in my head, and the info page descriptions above fit that fine;
I could not break out of that model.



> so if you plan
>to create more `level 1' backups, it is necessary to create a working
>copy of the snapshot file before running `tar'.  The above example will
>then be modified as follows:
>
>     $ cp /var/log/usr.snar /var/log/usr.snar-1
>     $ tar --create \
>                --file=archive.2.tar \
>                --listed-incremental=/var/log/usr.snar-1 \
>                /usr

It is this section here that was key; but it presents itself almost as a
side note for a case one would rarely encounter.  

An "incremental" backup can be interpreted as either:

  1) N-levels mode; changes made between each level N are captured
     or most recent previous level (N-1, N-2, ... 0).
  2) Bi-modal (full/incremental); incremental captures all changes 
     since the last full

I believe that creating several level 1 backups, one per day, would be
standard operation when performing incremental backups.  I was thinking it
might be more instructive to call out such usage scenarios more clearly by
modifying the explanation and examples.  The (abbreviated) examples below
show how to accomplish either of the above interpretations:

Example 1:  Creating a monthly full (level 0) backup, with daily (level 1)
  incrementals, and twice-daily (level 2) incrementals: 

# Create level 0 backup (all files)
$ tar --file=archive1.tar --listed-incremental=snap1.snar

# Prepare for and create a level 1 backup (all changes since level 0)
$ cp snap1.snar snap1.1.snar
$ tar --file=archive1.1.tar --listed-incremental=snap1.1.snar

# Prepare for and create a level 2 backup (all changes since 
# previous level 1)
$ cp snap1.snar snap1.1.1.snar
$ tar --file=archive1.1.1.tar --listed-incremental=snap1.1.1.snar

# Prepare for and create next level 2 backup (all changes since 
# previous level 2)
$ cp snap1.1.1.snar snap1.1.2.snar
$ tar --file=archive1.1.2.tar --listed-incremental=snap1.1.2.snar

# Prepare for and create next level 1 backup (all changes since
# previous level 1).
$ cp snap1.1.snar snap1.2.snar
$ tar --file=archive1.2.tar --listed-incremental=snap1.2.snar

# Create next level 0 backup (all files)
$ tar --file=archive2.tar --listed-incremental=snap2.snar


Example 2:  Creating a full backup and incrementals,
  where each incremental contains *all* changes since full

# Create full backup (all files)
$ tar --file=archive0.tar --listed-incremental=snap0.snar

# Prepare for first incremental backup (all changes since full)
$ cp snap0.snar snap1.snar
$ tar --file=archive1.tar --listed-incremental=snap1.snar

# Prepare for first incremental backup (again, all changes
  since full)
$ cp snap0.snar snap2.snar
$ tar --file=archive2.tar --listed-incremental=snap2.snar


If you agree with my feedback above, I'd be happy to provide the
documentation changes if desired.

Thanks for your time and support,
MrC