bug#6131: [PATCH]: fiemap support for efficient sparse file copy

[jeff.liu]

Jim Meyering wrote:
> Tao Ma wrote:
>> Hi Jim,
>> On 05/29/2010 12:44 AM, Jim Meyering wrote:
>>> Tao Ma wrote:
>>>
>>>> Hi Jim
>>>>
>>>> On 05/27/2010 06:30 PM, Jim Meyering wrote:
>>>>> jeff.liu wrote:
>>>>>> This is the revised version, it fixed the fiemap-start offset calculation
>>>>>> approach to remove it out
>>>>>> of the 'for (i = 0; i<   fiemap->fm_mapped_extents; i++)' loop.
>>>>> Hi Jeff,
>>>>>
>>>>> I've included below the state of my local changes.
>>>>> Unfortunately, with that 5-patch series, there is always a test failure
>>>>> on F13/ext4.  Maybe someone who knows more about extents can provide an
>>>>> explanation?
>>>> Just want to clarify why ocfs2 didn't work here. I guess the reason
>>>> also works for ext4 since both ext4 and ocfs2 use block group to
>>>> organize their blocks in the volume.
>>> Hi Tao,
>>>
>>> Thank you for the explanation.
>>> I'm glad to hear that there is no underlying problem.
>>>
>>>> I checked the perl test script to create sparse src file, it will
>>>> create contiguous bytes(around 20-24k) at an interval of around 40k.So
>>>> in general, these 20-24k should be contiguous. But that does exist
>>>> some scenario that they could be separately into 2 extents. Consider
>>>> one block group is used to allocate blocks to this file, when the
>>>> block group only has 10K left while you are requiring 20K, it will use
>>>> the left 10K in this group and allocate 10K from another block
>>>> group. That would become 2 extents since they can't be contiguous.
>>>> So I guess the right step is to check the holes by using filefrag if
>>>> you prefer this tool and want to make sure cp doesn't copy holes(I get
>>> Do you know of a tool other than filefrag that I could use?
>> nope.
>>> It looks like a small script could filter filefrag -v output, detect
>>> split extents and rewrite to make the output match what's expected.
>>> Probably not worth it, though, since this is already a very fragile test.
> 
> I went ahead and did it, after all.
> Here's the script, filefrag-extent-compare.
> With it, this test should pass when run on any of those four
> file system types.
> 
> 
> eval '(exit $?0)' && eval 'exec perl -wS "$0" ${1+"$@"}'
>   & eval 'exec perl -wS "$0" $argv:q'
>     if 0;
> # Determine whether two files have the same extents by comparing
> # the logical block numbers and lengths from filefrag -v for each.
> 
> # Invoke like this:
> # This helper function, f, extracts logical block number and lengths.
> # f() { awk '/^ *[0-9]/ {printf "%d %d ",$2,NF<5?$NF:$5} END {print ""}'; }
> # { filefrag -v j1 | f; filefrag -v j2 | f; } | ./filefrag-extent-compare
> 
> use warnings;
> use strict;
> (my $ME = $0) =~ s|.*/||;
> 
> my @line = <>;
> my $n_lines = @line;
> $n_lines == 2
>   or die "$ME: expected exactly two input lines; got $n_lines\n";
> 
> my @A = split ' ', $line[0];
> my @B = split ' ', $line[1];
> @A % 2 || @B % 2
>   and die "$ME: unexpected input: odd number of numbers; expected even\n";
> 
> my @a;
> my @b;
> foreach my $i (address@hidden/2-1) { $a[$i] = { L_BLK => $A[2*$i], LEN => 
> $A[2*$i+1] } };
> foreach my $i (address@hidden/2-1) { $b[$i] = { L_BLK => $B[2*$i], LEN => 
> $B[2*$i+1] } };
> 
> my $i = 0;
> my $j = 0;
> while (1)
>   {
>     !defined $a[$i] && !defined $b[$j]
>       and exit 0;
>     defined $a[$i] && defined $b[$j]
>       or die "address@hidden and address@hidden have different lengths, even 
> after adjustment\n";
>     ($a[$i]->{L_BLK} == $b[$j]->{L_BLK}
>      && $a[$i]->{LEN} == $b[$j]->{LEN})
>       and next;
>     ($a[$i]->{LEN} < $b[$j]->{LEN}
>      && exists $a[$i+1] && $a[$i]->{LEN} + $a[$i+1]->{LEN} == $b[$j]->{LEN})
>       and ++$i, next;
>     exists $b[$j+1] && $a[$i]->{LEN} == $b[$i]->{LEN} + $b[$i+1]->{LEN}
>       and ++$j, next;
>     die "differing extent:\n"
>       . "  [$i]=$a[$i]->{L_BLK} $a[$i]->{LEN}\n"
>       . "  [$j]=$b[$j]->{L_BLK} $b[$j]->{LEN}\n"
>   }
> continue
>   {
>     ++$i;
>     ++$j;
>   }
> 
> 
> 
Thanks Tao and Jim for helping this out!

-Jeff

-- 
With Windows 7, Microsoft is asserting legal control over your computer and is 
using this power to
abuse computer users.