bug#6131: [PATCH]: fiemap support for efficient sparse file copy

[Tao Ma]

Paul Eggert wrote:
> On 06/09/2010 11:56 PM, jeff.liu wrote:
>
>   
> > Yeah, I just realized that the behaviour I observed is caused by the delay 
> > allocation mechanism of
> > the particular FS.
> >     
>
> If the file system is using delayed allocation, then can
> the fiemap ioctl tell us that a file contains a hole (because nothing has been
> allocated there), but read() would tell us that the file contains nonzero data 
> at the same location
> (because it's sitting in a buffer somewhere)?  If so, we'd need to do something 
> like invoke
> fdatasync() on the file before issuing the fiemap ioctl, to force allocation; 
> or perhaps
> there's another ioctl that will do the allocation without having to actually do 
> a sync.
>   
actually there is a flag FIEMAP_EXTENT_DELALLOC in fiemap, and in this 
case, the file system
should return us a fiemap extent with this flag. I just did a simple 
test with ext4, and it looks that
it has a problem with sparse file. For a file like this:
dd if=/dev/zero of=/mnt/ext4/a bs=1M count=1
We can get a fiemap extent with DEALLOC successfully.
while with dd if=/dev/zero of=/mnt/ext4/a bs=1M count=1 seek=1
the ext4 can't return a valid fiemap extent.
Don't have time to dove into it yet.
> There's also the issue of copying from a file at the same time that some other 
> process
> is writing to it, but that is allowed to produce ill-defined behavior.  I'm 
> more worried
> about the case where some other process writes to the source file just before 
> 'cp' starts.
>   
If the file system can return the right DEALLOC fiemap, there should be 
no problem for it I guess.
> (Sorry, I haven't had time yet to dive into the proposed change; I'm still 
> trying to understand
> the environment.)
>
> One other thing: Solaris 10 supports lseek with the SEEK_HOLE and SEEK_DATA 
> options, which
> are easier to use and which (as far as I can tell from the manual) shouldn't 
> require anything
> fdatasync-ish.  Any objection if I propose support for that too?  It is 
> supposed to work
> with ZFS, something I can test here.
>   
I am afraid not. There are some discussions about this long time ago. 
See this article:
http://lwn.net/Articles/260795/
So it looks that SEEK_HOLE and SEEK_DATA are not welcomed by the kernel.

Regards,
Tao