Re: [Bug-tar] GNU tar generates malformed Pax attributes

[Pavel Raiskup]

Hi Jörg,

> On Monday, December 09, 2013 12:46:48 Joerg Schilling wrote:
> Pavel Raiskup <address@hidden> wrote:
>
> [...]
> >   $ setfattr -n user.test -v 0x01020304 FILE
> >   $ star H=exustar -xattr -c -f test.star FILE
>
> Well, this may be a problem that is a result from implementing a withdrawn
> draft on Linux.

This is problem in general: This is the fact  that user wants to have
extended attribute 'key=value' assigned to a FILE (which is ok) and the
value contains non-utf8 characters.  This is not about POSIX draft at
all..  at least afaik.  Or what I am missing here?

> [...]
>
> I don't see a problem with the nul character as it is a valid part of a 
> string.
> If other binary codes appear in an extended tar header (e.g. those that are
> used for UTF-8 encoding), this may be a real problem.

There is actually problem (and bsdtar is expectedly choking):

  The Open Group Base Specifications Issue 6 [1]:
  | An extended header shall consist of one or more records, each
  | constructed as follows:
  |
  | "%d %s=%s\n", <length>, <keyword>, <value>
  |
  | The extended header records shall be encoded according to the ISO/IEC
  | 10646-1:2000 standard (UTF-8).

.. and UTF-8 string may not contain nul character.

> > I don't think that it is a big disaster and the conclusion is that we
> > should stay backward compatible (which should not cost so much).
>
> Well, the main problem seems to be that gtar does not follow the method used 
> in
> star and archives implementation specific binary data even though there is a
> portable clean way to archive data for the withdrawn ACL draft.

The main problem is that there is no standardized way how to store binary
extended file attributes in pax Extended Header.

> With respect to SCHILY.xattr.system.posix_acl_access=.......... gtar is not
> compatible as this is a tag that always needs to be excluded.
>
> > > BTW: is this feature in use with gtar? I recently tried to compile gtar on
> > > Solaris and it does not seem to support ACLs even though the Changelog 
> > > claims
> > > such support. Given the fact that star includes portable ACL support since
> > > 2001 and Linux xattr support since 2003, I would guess that a typical 
> > > user of
> > > these features uses star instead of gtar.
> >
> > I am not able to test ACLs support on Solaris.  But apart from the numeric 
> > ACL
> > request [1] star/tar should be compatible in this regard.
>
> So you are running a system that does not allow to use VirtualBox?
>
> What do you understand by "numeric ACL request"?
>
> Just a note: tar is not compatible to star with regards to ACLs as Sun decided
> to use an implementation method that does not include numeric user IDs. The

I mean this (but completely off-topic regarding extended attributes)
thread:
http://article.gmane.org/gmane.comp.gnu.tar.bugs/5119/match=acl+numer

> ...
> > > Regarding the problem:
> > > It should be obvious that it is an implementation detail that Linux
> > > internally stores e.g. ACLs as "system xattrs" and that related data must
> > > not appear in an archive. Star of course excludes such data from the
> > > archive.
> >
> > Yes, but e.g. security.selinux is not excluded (yet?).
>
> If security.selinux was in use in 2003, I am sure that I discussed the problem
> with Andreas Grünbacher and we decided (based on his explanations) that this
> could be archived in raw mode.
>
> If this is something new, we need to discuss it as I of course cannot
> help if people implement features that might be in conflict with
> existing star archive format.
>
> Could you explain the problem?

I don't know whether this is new or not.  But you have the '\0' in raw
SELinux context now, e.g..  That is not a bug (even when I see that
trailing nul as redundant) in SELinux because you should not care about
exact content of that extended attribute (it is hidden behind the
'libselinux.so' wrappers, at least these days?).

The problem is: At least libarchive (bsdtar) does not expect the '\0' byte
in extended header.

> > ----
> >
> > This should be fixed somehow in GNU tar/star.  Maybe before that, we
> > should approximate to standardize that pax xattr storage finally to reach
> > some non capitalized keyword in pax header (Tim mentioned it some time
> > ago).  I mean linux/FreeBSD xattrs only, not the Solaris extended
> > attributes as that should be in future standardized separately and should
> > be discussed as Joerg said, not ACLs, not SELinux, ...
>
> If there are more than a few archives that include binary acl information
> inside linux xattrs, we may be forced to implement code to ignore these
> specific xattrs when in extract mode.

IMO, such attributes are worth ignoring.  At least GNU tar ignores all non
'user.*' extended attributes during extract.  Other that 'user.*'
attributes are usually system dependant and to extract them the user (a)
should be root /* linux at least */ and (b) should be sure about the
binary compatibility.

GNU tar atm stores now ACLs in 'SCHILY.acl.access' in text form (when
--acls is on) and in binary form (when --xattrs is on).

Yes; I think that this could be tweaked to not store ACLs in raw mode
(SCHILY.xattr.system.posix_acl_access) at least if --acls is on;  but when
users specify --xattrs during archiving, usually they expect that even ACLs
are backed up (they don't care; ACLs == xattrs..).  Not backing up the binary
ACLs may unexpectedly surprise user..

[1] http://pubs.opengroup.org/onlinepubs/009695299/utilities/pax.html

Pavel