Re: [Bug-tar] GNU tar generates malformed Pax attributes

[Joerg Schilling]

Pavel Raiskup <address@hidden> wrote:

> On Monday, December 09, 2013 00:02:41 Joerg Schilling wrote:
> > Tim Kientzle <address@hidden> wrote:
> >
> > > Pavel recently sent me an archive created with GNU tar
> > > that includes SCHILY.xattr extensions.
> >
> > I would guess that the problem is not SCHILY.xattr extensions in general but
> > a buggy implementation in gtar.
>
> Sadly, I tend to disagree.  The "SCHILY.xattr.attr" does not document encoding
> of binary data.  That means, something like that results in creating binary
> data in pax Extended Header (on Linux):

Not all features from star are completely documented already. In some cases, I 
recomment do RTSL.

I am e.g. not sure whether the method for multi-volume archives is fully 
documented, so people could reimplement it.

I am not sure whether I should write more documentation for the incremental 
dump feature - this includes the attribute tags in the extended tar header as 
well as the star restore symbol table. Given the fact that people seem to have 
frequent problems with restoring gtar incrementals, it may be one of the most 
important things to document in the future in cae the existing documentation is 
not sufficient.

In any case, I rely on feedback from users to be able to enhance the 
documentation.

>   $ setfattr -n user.test -v 0x01020304 FILE
>   $ star H=exustar -xattr -c -f test.star FILE

Well, this may be a problem that is a result from implementing a withdrawn 
draft on Linux.

The POSIX.1e draft has been withdrawn in 1997

Extended tar headers first appeared in 1997 with Solaris tar and have been 
standardized with POSIX.1-2001 and nobody did take care to think about the 
problem that somebody may come up, if someone likes to implement withdrawn 
features (that did not exist in OS implementations at that time) inside 
extended tar headers.


> > This is definitely a bug in gtar and I hope that not many archives exist
> > with this problem.
>
> According to that: At least archives coming from Red Hat distros (and possibly
> all distros having SELinux enabled) seem to produce bad archives (both with
> tar/star).  The problem is that SELinux contexts are stored NULL terminated
> (otherwise completely printable), the 'getxattr()' correctly returns the
> string with trailing '\0' and that is stored then in tar header.

I don't see a problem with the nul character as it is a valid part of a string.
If other binary codes appear in an extended tar header (e.g. those that are 
used for UTF-8 encoding), this may be a real problem.

> I don't think that it is a big disaster and the conclusion is that we
> should stay backward compatible (which should not cost so much).

Well, the main problem seems to be that gtar does not follow the method used in 
star and archives implementation specific binary data even though there is a 
portable clean way to archive data for the withdrawn ACL draft.

With respect to SCHILY.xattr.system.posix_acl_access=.......... gtar is not 
compatible as this is a tag that always needs to be excluded.

> > BTW: is this feature in use with gtar? I recently tried to compile gtar on
> > Solaris and it does not seem to support ACLs even though the Changelog 
> > claims
> > such support. Given the fact that star includes portable ACL support since
> > 2001 and Linux xattr support since 2003, I would guess that a typical user 
> > of
> > these features uses star instead of gtar.
>
> I am not able to test ACLs support on Solaris.  But apart from the numeric ACL
> request [1] star/tar should be compatible in this regard.

So you are running a system that does not allow to use VirtualBox?

What do you understand by "numeric ACL request"?

Just a note: tar is not compatible to star with regards to ACLs as Sun decided 
to use an implementation method that does not include numeric user IDs. The 
fact that thex repeated this bug in 2005 when introducing NFSv4 ACLs in "tar" 
caused the delay in the implemation of NFSv4 ACLs in star as I was first 
waiting for a fix in the library "libsec" and then run out of free time for 
such a project.

I cannot speak for gtar, as it does not compile in ACL support on my machie.

> > Regarding the problem:
> > It should be obvious that it is an implementation detail that Linux
> > internally stores e.g. ACLs as "system xattrs" and that related data must
> > not appear in an archive. Star of course excludes such data from the
> > archive.
>
> Yes, but e.g. security.selinux is not excluded (yet?).

If security.selinux was in use in 2003, I am sure that I discussed the problem 
with Andreas Grünbacher and we decided (based on his explanations) that this 
could be archived in raw mode.

If this is something new, we need to discuss it as I of course cannot help if 
people implement features that might be in conflict with existing star archive 
format. 

Could you explain the problem?


> ----
>
> This should be fixed somehow in GNU tar/star.  Maybe before that, we
> should approximate to standardize that pax xattr storage finally to reach
> some non capitalized keyword in pax header (Tim mentioned it some time
> ago).  I mean linux/FreeBSD xattrs only, not the Solaris extended
> attributes as that should be in future standardized separately and should
> be discussed as Joerg said, not ACLs, not SELinux, ...

If there are more than a few archives that include binary acl information 
inside linux xattrs, we may be forced to implement code to ignore these 
specific xattrs when in extract mode.

There are many problems that could be avoided in case people would ask before 
implementing things that are in a relation to backups.

Think e.g. about the problem with Linux file flags, that cannot be archived for 
device nodes just because the Linux kernel interface requires to open the 
related file and to run an ioctl() on that fd. This could rewind a tape or 
cause other unwanted device specific actions and this in general makes star 
slow when archiving file flags on Linux.

For the same reason, I am unhappy with the way Solaris userland code retrieves 
information on the existence of non-trivial ACLs and on the existence of 
extended attributes as this makes star slower than needed. 

Jörg

-- 
 EMail:address@hidden (home) Jörg Schilling D-13353 Berlin
       address@hidden                (uni)  
       address@hidden (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily