bug#6131: [PATCH]: fiemap support for efficient sparse file copy

[Jim Meyering]

Sunil Mushran wrote:
> Jim Meyering wrote:
>>> Hi Jeff,
>>>
>>> I've included below the state of my local changes.
>>> Unfortunately, with that 5-patch series, there is always a test failure
>>> on F13/ext4.  Maybe someone who knows more about extents can provide an
>>> explanation?
>>>
>>> Here's a small example to demonstrate:
>>>
>>> Create a file with many extents:
>>>
>>>      perl -e 'BEGIN { $n = 19 * 1024; *F = *STDOUT }' \
>>>        -e 'for (1..100) { sysseek (*F, $n, 1)' \
>>>        -e '&&  syswrite (*F, "."x$n) or die "$!"}'>  j1
>>>
>>> Using the patched "cp", repeat the following 10 or 20 times:
>>>
>>>      ./cp --sparse=always j1 j2; sync
>>>      filefrag -v j1 | awk '/^ / {print $1,$2}'>  ff1 || fail=1
>>>      filefrag -v j2 | awk '/^ / {print $1,$2}'>  ff2 || fail=1
>>>      diff -u ff1 ff2 || fail=1
>>>
>>> Usually there is no diff output, but occasionally it'll print this:
>>> [hmm... today it consistently prints these differences every other time.]
>
> The reason it does not work is because the sparse file created by cp
> may not be sparse (or sparse enough). And that is because cp reads
> is chunks of st_blocksize and skips the write only if the entire chunk
> is zero. The perl script creates the file in 19K chunks (alternate writes
> and holes).

Thanks for replying.

However, your description of how GNU cp works suggests that you're
looking at the pre-FIEMAP semantics.  Please refer to the patches here

    http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/20534/focus=20630

that make it use FIEMAP.

> So on a 4K fs, the file created by the script will have 4 blocks as holes
> (avg). But when cp makes it, it could fill out those holes because the read
> granularity could be coarser. For example, ocfs2 fills out st_blocksize
> with the fs cluster size (allocation size) which could be larger than the
> block size.
>
> My suggestion is to not use filefrag but to use md5sum to compare the two
> files.

That would be pointless.
The goal of the test is to determine that the FIEMAP copy
did indeed preserve the extents.  If I do as you suggest, even if
cp mistakenly filled in all holes, the test would still pass.