bug#6131: [PATCH]: fiemap support for efficient sparse file copy

[Jim Meyering]

jeff.liu wrote:

> Jim Meyering wrote:
>> Jim Meyering wrote:
>>> Subject: [PATCH 01/10] cp: Add FIEMAP support for efficient sparse file copy
>>
>> FYI, using those patches, I ran a test for the first time in a few days:
>>
>>     check -C tests TESTS=cp/sparse-fiemap VERBOSE=yes
>>
>> It failed like this on an ext4 partition using F13:
>>
>>     + timeout 10 cp --sparse=always sparse fiemap
>>     + fail=1
>>     ++ stat --printf %s sparse
>>     ++ stat --printf %s fiemap
>>     + test 1099511628800 = 0
>>     + fail=1
>>
>> That is very odd.  No diagnostic from cp, yet it failed
>> after creating a zero-length file.
>>
>> Here's the corresponding piece of the script:
>>
>>     # It takes many minutes to copy this sparse file using the old method.
>>     # By contrast, it takes far less than 1 second using FIEMAP-copy.
>>     timeout 10 cp --sparse=always sparse fiemap || fail=1
>>
>>     # Ensure that the sparse file copied through fiemap has the same size
>>     # in bytes as the original.
>>     test $(stat --printf %s sparse) = $(stat --printf %s fiemap) || fail=1
>>
>> However, so far I've been unable to reproduce the failure,
>> running hundreds of iterations:
>>
>>     for i in $(seq 300); do printf .; make check -C tests \
>>       TESTS=cp/sparse-fiemap VERBOSE=yes >& makerr-$i || break; done
>>
>> Have any of you heard of a problem whereby a cold cache can cause
>> such a thing?  "echo 3 > /proc/sys/vm/drop_caches" didn't help.
> Hi Jim,
>
> Have you run `sync' before clean the buffer and caches?  Actually, even run 
> `sync' first, sometimes,
> maybe the dirty objects still can not be freed in some cases. :(

Hi Jeff,

Thanks for the feedback.

The test script I ran above does not invoke sync between
the dd invocation and the cp --sparse.
If running sync before cp is required in order to avoid a failure,
then I consider that a bug in cp.

> I can reproduce this issue on ext4 and btrfs(physical mounted partition) or 
> just run the

If you can reproduce the cp failure, then please use my latest
patch that diagnoses 2nd and subsequent FIEMAP ioctl failure
and tell me what cp prints when it fails.

I would really like to know how the first ioctl can succeed,
yet the second one fails.

Your demonstrations show that *after* the cp, one may have
to sync in order to see precisely if/how the newly-created
destination file is fragmented, but that is nothing new.
Note the commented-out "sync" uses in the test script.
As far as I can see, your examples do not show cp failing
like it did for me.