Re: [rdiff-backup-users] Empty directory trees not being backed up

[Toni Price]

Ben Escoto wrote:

> > Are you sure its a bug?  Maybe it's just some unexpected
> > --include/--exclude interaction, or a permission problem?
> >  

Ok, I have managed to repeat the problem on some dummy data I've created.

I've attached a copy of the data with a bash script to illustrate what's 
happening ... this data (sort of) mimics my own data structure, which is 
why I've done it like this. (If you untar it and run 
test-rdiff-backup.sh from the directory where it's been untarred, it 
should copy some data to /tmp/rdiff-data and run two backups to 
/tmp/rdiff-test/bak).

It's entirely possible that I simply don't understand properly how the 
--includes and --excludes work, but this behaviour seems odd to me.

My data structure looks as follows:

rdiff-data/
|-- docs
|   |-- technotes
|   |   |-- anotherdir
|   |   |   |-- empty_tree_2
|   |   |   |   |-- dir1
|   |   |   |   |   `-- structure
|   |   |   |   |       |-- sub1
|   |   |   |   |       `-- sub2
|   |   |   |   |           `-- subsub
|   |   |   |   `-- dir2
|   |   |   |       |-- lower1
|   |   |   |       `-- lower2
|   |   |   `-- somefile.txt
|   |   |-- empty_tree_1
|   |   |   `-- sub1
|   |   |       |-- sibling1
|   |   |       |   `-- nextdir
|   |   |       `-- sibling2
|   |   `-- misc
|   |       |-- dir1
|   |       |   `-- fox-and-dogs.txt
|   |       `-- dir2
|   |           `-- sub1
|   |               `-- stuff.txt
|   |-- tmp
|   |   |-- tempdir1
|   |   |-- tempfile1.txt
|   |   `-- tempfile2.txt
|   `-- various
|       |-- another
|       |   |-- empty_tree_3
|       |   |   `-- testdir1
|       |   |       `-- testdir2
|       |   |           |-- testsub1
|       |   |           `-- testsub2
|       |   `-- moreStuff.txt
|       |-- file1.txt
|       `-- file2.txt
|-- mediafiles
|   `-- audio
|       |-- Classical
|       |   |-- audio1
|       |   `-- audio2
|       |-- Jazz
|       |   `-- audio
|       `-- Playlists
|           |-- playlist1.txt
|           `-- playlist2.txt
|-- ref
|   `-- ref.txt
`-- software
   `-- sofware-stuff.txt

37 directories, 15 files

Within this there are three empty directory trees,
 rdiff-data/docs/technotes/empty_tree_1/
 rdiff-data/docs/technotes/anotherdir/empty_tree_2/
 rdiff-data/docs/various/another/empty_tree_3

Basically, if I run the following command:

/usr/bin/rdiff-backup \
--exclude /tmp/rdiff-data/docs/tmp/'*' \
--include /tmp/rdiff-data/mediafiles/audio/Playlists \
--exclude /tmp/rdiff-data/mediafiles/audio/'*' \
--exclude /tmp/rdiff-data/ref/'*' \
--exclude /tmp/rdiff-data/software/'*' \
/tmp/rdiff-data /tmp/rdiff-test/bak/data1

it works as expected, whilst if I change it as follows (the change I've 
made here is from /tmp/rdiff-data to '**' in the --include)

/usr/bin/rdiff-backup \
--exclude /tmp/rdiff-data/docs/tmp/'*' \
--include '**'/mediafiles/audio/Playlists \
--exclude /tmp/rdiff-data/mediafiles/audio/'*' \
--exclude /tmp/rdiff-data/ref/'*' \
--exclude /tmp/rdiff-data/software/'*' \
/tmp/rdiff-data /tmp/rdiff-test/bak/data2

then it all goes a bit weird, and excludes empty_tree_1, empty_tree_2 & 
empty_tree_3 from the backup (as well as excluding tmp, ref and software 
altogether, which is not my intention).

Here's the output I get when running test-rdiff-backup.sh. (Hopefully it 
will produce the same results when run in a different environment):

===== START output =================================================

Files only in /tmp/rdiff-data after [correct] backup:

Only in /tmp/rdiff-data/docs/tmp: tempdir1
Only in /tmp/rdiff-data/docs/tmp: tempfile1.txt
Only in /tmp/rdiff-data/docs/tmp: tempfile2.txt
Only in /tmp/rdiff-data/mediafiles/audio: Classical
Only in /tmp/rdiff-data/mediafiles/audio: Jazz
Only in /tmp/rdiff-data/ref: ref.txt
Only in /tmp/rdiff-data/software: sofware-stuff.txt


Files only in /tmp/rdiff-test/bak/data1 after [correct] backup:

Only in /tmp/rdiff-test/bak/data1: rdiff-backup-data

------------------------------------------------

Files only in /tmp/rdiff-data after [incorrect] backup:

Only in /tmp/rdiff-data/docs/technotes/anotherdir: empty_tree_2
Only in /tmp/rdiff-data/docs/technotes: empty_tree_1
Only in /tmp/rdiff-data/docs: tmp
Only in /tmp/rdiff-data/docs/various/another: empty_tree_3
Only in /tmp/rdiff-data/mediafiles/audio: Classical
Only in /tmp/rdiff-data/mediafiles/audio: Jazz
Only in /tmp/rdiff-data: ref
Only in /tmp/rdiff-data: software


Files only in /tmp/rdiff-test/bak/data2 after [incorrect] backup:

Only in /tmp/rdiff-test/bak/data2: rdiff-backup-data

===== END output ==================================================

What I don't understand is, why would the following include:
--include '**'/mediafiles/audio/Playlists
cause rdiff-backup to exclude something like 
"/tmp/rdiff-data/docs/technotes/anotherdir/empty_tree_2"? Am I simply 
missing something about how --include should work?

Thanks ...
Toni

rdiff-test.tar.gz
Description: application/gzip