Re: [Bug-tar] Tar 1.27 has become non-deterministic

[Nathan Stratton Treadway]

On Sun, Nov 03, 2013 at 07:15:55 -0500, Nathan Stratton Treadway wrote:
> On Sun, Nov 03, 2013 at 09:27:04 +1100, David Barri wrote:
> > Sure. Here's a diff between two tars I created using tar -cp -T filelist
> > 
> > > diff -u <(xxd a.tar) <(xxd b.tar)
> > --- /proc/self/fd/11 2013-11-03 09:21:32.144102315 +1100
> > +++ /proc/self/fd/12 2013-11-03 09:21:32.147435636 +1100
> > @@ -188775,7 +188775,7 @@
> >  02e1660: 0000 0000 3030 3030 3634 3400 3030 3030  ....0000644.0000
> >  02e1670: 3030 3000 3030 3030 3030 3000 3030 3030  000.0000000.0000
> >  02e1680: 3030 3030 3136 3100 3132 3233 3530 3236  0000161.12235026
> > -02e1690: 3436 3300 3031 3136 3433 0020 4c00 0000  463.011643. L...
> > +02e1690: 3531 3100 3031 3136 3335 0020 4c00 0000  511.011635. L...
> 
> Are these both created with tar 1.27?  If so, how does the v1.26 file
> look in those same spots?

Actually, I think I see the problem...

The "L" near the end of the changed line is a flag that indicates that
the archive member in question is of type "GNUTYPE_LONGNAME".  All of
the blocks show in your "diff" message show that flag, so it would seem
that the issue is confined to the handling of members with long names
(i.e. over 100 characters long).

The tar archive header for such files is written by the
create.c:write_gnu_long_link()  function, and comparing that function
between 1.26 and 1.27 I see:

-  header = start_private_header ("././@LongLink", size, time (NULL));
-  FILL (header->header.mtime, '0');
-  FILL (header->header.mode, '0');
-  FILL (header->header.uid, '0');
-  FILL (header->header.gid, '0');
-  FILL (header->header.devmajor, 0);
-  FILL (header->header.devminor, 0);
+  header = start_private_header ("././@LongLink", size, start_time.tv_sec);


Presumably the idea was that those fields are all already set to their
"default" value by the start_private_header() call, and thus there's no
reason to explicitly clear them... but in this case the removal of the
  FILL (header->header.mtime, '0');
line does mean that the header generated in v1.26 had a blank mtime
field, while in 1.27 the value found there is based on the current time
(as of the start of this tar run)....


Anyway, I was able to reproduce this with a 101-character filename:

  $ echo hi > 
12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901
  $ tar -cp -f longname_1.27.tar 
12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901
  

Comparing that with a tar file generated using v1.26 (using
  $ diff -U 9  <(xxd longname_1.26.tar) <(xxd longname_1.27.tar)
), I get:

 0000000: 2e2f 2e2f 404c 6f6e 674c 696e 6b00 0000  ././@LongLink...
 0000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 0000020: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 0000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 0000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 0000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
-0000060: 0000 0000 3030 3030 3030 3000 3030 3030  ....0000000.0000
+0000060: 0000 0000 3030 3030 3634 3400 3030 3030  ....0000644.0000
 0000070: 3030 3000 3030 3030 3030 3000 3030 3030  000.0000000.0000
-0000080: 3030 3030 3134 3600 3030 3030 3030 3030  0000146.00000000
-0000090: 3030 3000 3031 3135 3636 0020 4c00 0000  000.011566. L...
+0000080: 3030 3030 3134 3600 3132 3233 3534 3533  0000146.12235453
+0000090: 3330 3000 3031 3136 3430 0020 4c00 0000  300.011640. L...
 00000a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 0000100: 0075 7374 6172 2020 0072 6f6f 7400 0000  .ustar  .root...
 0000110: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 0000120: 0000 0000 0000 0000 0072 6f6f 7400 0000  .........root...



                                                        Nathan






 



----------------------------------------------------------------------------
Nathan Stratton Treadway  -  address@hidden  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239