Re: [Bug-tar] delete/append corruption on busybox tarfiles

[Philip Rowlands]

On Thu, 16 Aug 2007, Philip Rowlands wrote:

> Yes, there is a difference. Deleting the first archive member (a directory, if 
> it matters), results in an file full of 10240 nulls, which corresponds to the 
> default --blocking-factor.

I'm being dumb - the default recursion is doing this. --no-recursion 
shows a different issue.

I did a little more digging, and can now reproduce this problem using 
only GNU tar.

To generate a test dataset for archival, I used this:

$ mkdir seqout
$ for f in $(seq 100 500); do seq -f "file $f - line %g" $f > seqout/$f; done

In the case where GNU tar's --delete leads to errors, the deleted first 
member is exactly this (from busybox):

00000000  73 65 71 6f 75 74 2f 00  00 00 00 00 00 00 00 00  |seqout/.........|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000060  00 00 00 00 30 30 34 30  37 35 30 00 30 30 32 33  |....0040750.0023|
00000070  34 35 36 00 30 30 31 31  36 31 30 00 30 30 30 30  |456.0011610.0000|
00000080  30 30 30 30 30 30 30 00  31 30 36 36 31 33 34 32  |0000000.10661342|
00000090  30 33 37 00 30 31 30 35  32 36 00 20 35 00 00 00  |037.010526. 5...|
000000a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000100  00 75 73 74 61 72 20 20  00 70 68 72 00 00 00 00  |.ustar  .phr....|
00000110  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000120  00 00 00 00 00 00 00 00  00 65 6e 67 00 00 00 00  |.........eng....|
00000130  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000140  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000160  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000170  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000180  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000190  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

The only diff against GNU tar being

-00000060  00 00 00 00 30 30 34 30  37 35 30 00 30 30 32 33  |....0040750.0023|
+00000060  00 00 00 00 30 30 30 30  37 35 30 00 30 30 32 33  |....0000750.0023|

-00000090  30 33 37 00 30 31 30 35  32 36 00 20 35 00 00 00  |037.010526. 5...|
+00000090  30 33 37 00 30 31 30 35  32 32 00 20 35 00 00 00  |037.010522. 5...|

which corresponds to an additional S_IFDIR bit being set in the mode 
field, with checksum adjustment.

At the end of the file, the only difference is the number of appended 
null blocks. The final member requires 352 bytes of null-padding to 
complete the 512-byte block, after which busybox appends 1024 bytes (2 
blocks) of further nulls, whereas GNU tar appends 2048 bytes (4 blocks) 
to bring the archive output up to a total size of 512*20*261, in line 
with the default --blocking-factor.

I now restate the problem using only GNU tar:

$ ./gtar-1.18 --create --blocking-factor=2 --file gnutar-b2.tar seqout

$ ./gtar-1.18 --file gnutar-b2.tar --list | wc -l
402

# --blocking-factor=20 is the default, delete first member
$ ./gtar-1.18 --file gnutar-b2.tar --no-recursion --delete seqout

# 2 members removed, errors now issued
$ ./gtar-1.18 --file gnutar-b2.tar --list | wc -l
./gtar-1.18: Skipping to next header
./gtar-1.18: Error exit delayed from previous errors
400

# alternatively, along the lines of the original report
$ ./gtar-1.18 --create --blocking-factor=2 --file gnutar-b2.tar seqout

# delete middle member
$ ./gtar-1.18 --file gnutar-b2.tar --delete seqout/350

# attempt to re-add
$ ./gtar-1.18 --file gnutar-b2.tar --append seqout/350
./gtar-1.18: Skipping to next header
./gtar-1.18: Error exit delayed from previous errors


Cheers,
Phil