Re: [Bug-tar] use optimal file system block size

[Andreas Dilger]

On Jul 18, 2018, at 9:03 AM, Ralph Corderoy <address@hidden> wrote:
> 
> Hi Christian,
> 
>> $ stat -c %o data/blob
>> 2097152
> ...
>> **tar** does not explicitly use the block size of the file system
>> where the files are located, but, for a reason I don't know (feel free to 
>> educate me), 10 KiB:
> 
> Historic, that being 20 blocks where a block is 512 B.  See `Blocking
> Factor'.  https://www.gnu.org/software/tar/manual/tar.html#SEC160
> 
> It can be changed.
> 
>    $ strace -e write -s 10 tar cbf 4096 foo.tar foo
>    write(3, "foo\0\0\0\0\0\0\0"..., 2097152) = 2097152
>    +++ exited with 0 +++
>    $
> 
>> I would like to propose to use the native file system block size in
>> favor of the currently used 10 KiB.
> 
> I can't see the default changing.  POSIX's pax(1) states for ustar
> format that the default for character devices is 10 KiB, and allows for
> multiples of 512 up to an including 32,256.  So you're suggesting the
> default is to produce an incompatible tar file.

The IO size from the storage does not need to match the recordsize
of the tar file.  It may be that writing to an actual tape character
device needs to use 10KB writes, but for a regular file on a block
device (which is 99% of tar usage) it can still write 10KB records,
but just write a few hundred of them at a time.

What network filesystem are you using?  Typically, such small IOPS
should be hidden from the filesystem with readahead and writeback
cache, though of course there is still more overhead from having
lots of system calls.

Cheers, Andreas

signature.asc
Description: Message signed with OpenPGP