Re: [Bug-tar] [PATCH] tar: Fixed bug, tar always updates the archive when path ends with a trailing slash

[Andrey Arapov]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hi folks,

I have corrected a little typo in comment :-)

Signed-off-by: Andrey Arapov <address@hidden>
diff -u src/names.c.orig src/names.c
- --- src/names.c.orig  2014-03-26 21:58:48.000000000 +0100
+++ src/names.c 2015-10-19 10:11:10.009126520 +0200
@@ -705,6 +705,10 @@
 
   for (p = namelist; p; p = p->next)
     {
+      /* zap a trailing slash */
+      if (ISSLASH(p->name[length]))
+        p->name[length] = '\0';
+
       if (p->name[0]
          && exclude_fnmatch (p->name, file_name, p->matching_flags))
        return p;


I have also found that tar does not strip unnecessary characters,
e.g. leading './', '/' and trailing '/'. This is what causes tar to
constantly update the archive.
Well, probably with './' this is a "feature" as someone might want to
explicitly keep them in their archives, however this can cause unnecessary
data duplication and uglinness in the archive (see below).
The leading '/' causes tar to always update the archive, even when files
have not changed.


1) Creating directory structure and initializing the archive

address@hidden:~/Downloads/src/tar-1.28/test1$ mkdir -p dir1/dir2
address@hidden:~/Downloads/src/tar-1.28/test1$ touch dir1/file1
address@hidden:~/Downloads/src/tar-1.28/test1$ touch dir1/dir2/file2

address@hidden:~/Downloads/src/tar-1.28/test1$ ../src/tar --update -vf a.tar -T 
- <<<$(echo "dir1")
dir1/
dir1/dir2/
dir1/dir2/file2
dir1/file1


2) Updating the archive

address@hidden:~/Downloads/src/tar-1.28/test1$ ../src/tar --update -vf a.tar -T 
- <<<$(echo "dir1")
address@hidden:~/Downloads/src/tar-1.28/test1$ 

Result: as expected, tar does not update the archive as files have not been 
modified.


3) Updating archive. filelist (-T) has path name with a trailing slash (my 
patch fixes that now)

address@hidden:~/Downloads/src/tar-1.28/test1$ ../src/tar --update -vf a.tar -T 
- <<<$(echo "dir1/")
dir1/
dir1/dir2/
dir1/dir2/file2
dir1/file1

Result: unexpected. Tar compares names of files as "dir1/" (passed with -T) == 
"dir1" (read from tar archive)


4) Updating archive. filelist (-T) has path name with leading './'

address@hidden:~/Downloads/src/tar-1.28/test1$ ../src/tar --update -vf a.tar -T 
- <<<$(echo "./dir1")
./dir1/
./dir1/dir2/
./dir1/dir2/file2
./dir1/file1

Result: unexpected. Why to distinct files starting with './' ? This causes data 
duplication.


5) Updating archive. filelist (-T) has path name starting with leading '/' 
causes to _always_ update the archive.

address@hidden:~/Downloads/src/tar-1.28/test1$ ../src/tar --update -vf a.tar -T 
- <<<$(echo "/home/arno/Downloads/src/tar-1.28/test1/dir1")
../src/tar: Removing leading `/' from member names
/home/arno/Downloads/src/tar-1.28/test1/dir1/
/home/arno/Downloads/src/tar-1.28/test1/dir1/dir2/
/home/arno/Downloads/src/tar-1.28/test1/dir1/dir2/file2
/home/arno/Downloads/src/tar-1.28/test1/dir1/file1
address@hidden:~/Downloads/src/tar-1.28/test1$ ../src/tar --update -vf a.tar -T 
- <<<$(echo "/home/arno/Downloads/src/tar-1.28/test1/dir1")
../src/tar: Removing leading `/' from member names
/home/arno/Downloads/src/tar-1.28/test1/dir1/
/home/arno/Downloads/src/tar-1.28/test1/dir1/dir2/
/home/arno/Downloads/src/tar-1.28/test1/dir1/dir2/file2
/home/arno/Downloads/src/tar-1.28/test1/dir1/file1

Result: unexpected.
Reason: tar compares '/path/dir' (passed with -T) with 'path/dir' (read from 
the archive)
Workaround: to use "-P, --absolute-names" flag


6) After the above tests, the archive contents look pretty messy now

address@hidden:~/Downloads/src/tar-1.28/test1$ tar tf a.tar 
dir1/
dir1/dir2/
dir1/dir2/file2
dir1/file1
dir1/
dir1/dir2/
dir1/dir2/file2
dir1/file1
./dir1/
./dir1/dir2/
./dir1/dir2/file2
./dir1/file1
home/arno/Downloads/src/tar-1.28/test1/dir1/
home/arno/Downloads/src/tar-1.28/test1/dir1/dir2/
home/arno/Downloads/src/tar-1.28/test1/dir1/dir2/file2
home/arno/Downloads/src/tar-1.28/test1/dir1/file1
home/arno/Downloads/src/tar-1.28/test1/dir1/
home/arno/Downloads/src/tar-1.28/test1/dir1/dir2/
home/arno/Downloads/src/tar-1.28/test1/dir1/dir2/file2
home/arno/Downloads/src/tar-1.28/test1/dir1/file1


kind regards,
Andrey Arapov


October 19 2015 12:14 AM, "Andrey Arapov" <address@hidden> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> Dear developers,
> 
> seems there is a bug which causes tar to always update the contents of an 
> archive if the directory
> ends with a slash in the FILE specified with "-T" flag,
> see below the example of this unexpected behavior and a patch that I made:
> 
> - -- I've tried both version of tar
> 
> address@hidden:~$ tar --version
> tar (GNU tar) 1.27.1
> address@hidden:~/Downloads/src/tar-1.28$ ./src/tar --version
> tar (GNU tar) 1.28
> 
> - -- Normal and expected behavior
> 
> address@hidden:~$ /bin/tar --create -vf archive.tar .ssh
> .ssh/
> .ssh/id_ed25519
> .ssh/id_ed25519.pub
> .ssh/known_hosts
> address@hidden:~$ /bin/tar --update -vf archive.tar .ssh
> address@hidden:~$ /bin/tar --update -vf archive.tar .ssh/
> 
> (with -v flag tar would output whether there is a "new" file go into an 
> archive.tar)
> 
> - -- Normal and expected behavior (using file list)
> 
> address@hidden:~$ cat list.txt
> .ssh
> address@hidden:~$ /bin/tar --update -vf archive.tar -T list.txt
> address@hidden:~$ /bin/tar --update -vf archive.tar -T list.txt
> 
> - -- BUG: Adding slash in the name of path (.ssh => .ssh/), causes tar to 
> update the archive always
> 
> address@hidden:~$ vim list.txt
> address@hidden:~$ cat list.txt
> .ssh/
> address@hidden:~$ /bin/tar --update -vf archive.tar -T list.txt
> .ssh/
> .ssh/id_ed25519
> .ssh/id_ed25519.pub
> .ssh/known_hosts
> address@hidden:~$ /bin/tar --update -vf archive.tar -T list.txt
> .ssh/
> .ssh/id_ed25519
> .ssh/id_ed25519.pub
> .ssh/known_hosts
> address@hidden:~$ tar -tvf archive.tar
> drwx------ arno/arno 0 2015-10-12 23:48 .ssh/
> - -rw------- arno/arno 444 2015-10-14 20:36 .ssh/id_ed25519
> - -rw-r--r-- arno/arno 93 2015-07-29 21:59 .ssh/id_ed25519.pub
> - -rw-r--r-- arno/arno 3842 2015-10-15 20:51 .ssh/known_hosts
> drwx------ arno/arno 0 2015-10-12 23:48 .ssh/
> - -rw------- arno/arno 444 2015-10-14 20:36 .ssh/id_ed25519
> - -rw-r--r-- arno/arno 93 2015-07-29 21:59 .ssh/id_ed25519.pub
> - -rw-r--r-- arno/arno 3842 2015-10-15 20:51 .ssh/known_hosts
> drwx------ arno/arno 0 2015-10-12 23:48 .ssh/
> - -rw------- arno/arno 444 2015-10-14 20:36 .ssh/id_ed25519
> - -rw-r--r-- arno/arno 93 2015-07-29 21:59 .ssh/id_ed25519.pub
> - -rw-r--r-- arno/arno 3842 2015-10-15 20:51 .ssh/known_hosts
> 
> - -- The patch
> 
> I have decided to try myself and make a patch, please review it carefully, I 
> do not want to break
> anything :-)
> 
> Signed-off-by: Andrey Arapov <address@hidden>
> diff -u src/names.c.orig src/names.c
> - --- src/names.c.orig 2014-03-26 21:58:48.000000000 +0100
> +++ src/names.c 2015-10-18 23:58:52.131318666 +0200
> @@ -705,6 +705,10 @@
> 
> for (p = namelist; p; p = p->next)
> {
> + /* only zap trailing a slash */
> + if (ISSLASH(p->name[length]))
> + p->name[length] = '\0';
> +
> if (p->name[0]
> && exclude_fnmatch (p->name, file_name, p->matching_flags))
> return p;
> 
> kind regards,
> Andrey Arapov
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
> 
> iQIcBAEBCAAGBQJWJBj3AAoJEDaN2i6aRx6s7iwP/34JQbDQhFnM8O1upuJg77+E
> qstMscV5BtFJ1eHWhmaDPQrSz2Z6zRGuYPEYZ7K8rtr6d64XZqS8/IPSBZxl4O3d
> +Lqk390Gxh9gDbsDD2vaJrgE2UGrBxvdFLLHHkBME0c+ksaMZP3pl8ui9wvaybsM
> H/tecz3SjjAHbUyzV60qk79p231hKO/wIN8dJxXymKJUrQ4WLzRjhOcRLw35WqvS
> ASGWUq1C0JlEMWeL0rPRGucKOgsDjJh7SUaWeMGW6wTSTbEE1bbTKy0kEfNjFRu1
> nL7i24I0zxNFNb37eoBPfLMLvRXjI5tbaPvOl7MarwIfDiuOjKUbuO2Zm9lLUNDb
> 7fA8tJxxoYgI08vUN0LKMYNF3zQc+ETHu1yK+cb4y3ZAVhSbnVox4e2qfluXEu/U
> S8L1BETLFKQ8OVafVoAojI4rSc829Dn2SR7Pdr2WzEoGpfDFrTvq8EbiB4QppRGE
> 3E55AZryHEet+54xH1bZcIG8Ty5SB5i8hqu+cgSEMtemc4uplxbWGENV+giOtbVH
> JlhtwCa23WijD32jWCFLXeUChdNlO22W2V+IvaVPEZ7sMYnIt2N5ZNCwIQp1TvT5
> eeBHHXJ+rpN7emFJoL7nqy5Fg/2qeSzPSWeDWWiLrhThwvtDOKzSL6BbWLrFVLLk
> 1YFyA3MrMS2xrXgbxwEK
> =ux7W
> -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCgAGBQJWJKuoAAoJEDaN2i6aRx6sX9oP/A6VF8WPuWoU8YBneD9AiA5A
IbTuoHbniHPLtStnYpNDJPO3s3SvPdWzt1Ue+luLej1peq89vIaOjydh9kVvvILd
x2hrvVuJUKUUFgvz782C4bDi9jH0gRZ6jhqwfacP1UdZGVTrwdXJPGT83mJASXco
86w6nOyNhmhWXj0b4vIftusBRGyVwZdcYMUSXynxFZMdPcrwtxyPkaQfpW2Y4sc3
T+fhplAaBC1Ifh4j4NzOQGoN1DWIyhtjrBjbWCjGGkPMnbv5NjtB4hfmpDiwPJAE
Yi6H3xiZ9X7pjy4yXKTapFSF1JPHvUtT5AaSapI9xJ/Iez6qUdLRhUvUEdopNBMR
HksbchBbuuP09GDbBoLO43uvI1fPjWJGVsKRxApqHIqRicoIxQKAmmUlHdzroMYV
cX914SA+UE63Ca7yULJ3Gu693uZgT0wnWAZO27SDnaSiA9rLIM48gBA9l09Cg48c
2qCEYjmHByz15Y1FjkkpVxJhZY4CEhiNmX+jiEIAOJc+ugoBKo8iL6XyG6zZulYX
1bgL0qZbktwi2yI6k97P86LWsixVZUQ2Uh0z68EzGRvDLB2CWovJloLDF5PgUoyo
5cPFM/BRJuEKVD+B//lXW+xsJvEgDe8+0iEqbNH0SyedxJCnYtJONtvI8DaJk3iC
UxK9l4vJfC4tIBEZoaGM
=fLMi
-----END PGP SIGNATURE-----