Re: RFC: new cp option: --efficient-sparse=HOW

[Eric Blake]

On 01/31/2011 02:46 PM, Jim Meyering wrote:
> Now that we have can read sparse files efficiently,
> what if I want to copy a 20PiB sparse file, and yet I want to
> be sure that it does so efficiently.  Few people can afford
> to wait around while a normal processor and storage system process
> that much raw data.  But if it's a sparse file and the src and dest
> file systems have the right support (FIEMAP ioctl), then it'll be
> copied in the time it takes to make a few syscalls.
> 
> Currently, when the efficient sparse copy fails, cp falls back
> on the regular, expensive, read-every-byte approach.
> 
> This proposal adds an option, --efficient-sparse=required,
> to make cp fail if the initial attempt to read the sparse file fails,
> rather than resorting to the regular (very slow in the above case) copy
> procedure.
> 
> The default is --efficient-sparse=auto, and for symmetry,
> I've provided --efficient-sparse=never, in case someone finds
> a reason to want to skip the ioctl.

Conversely, what happens if I have a file that contains large blocks of
zeros but is NOT fully sparse (plausible, since we're still facing the
fact that it is still not easy to punch holes into existing files when
data in that portion of the file is no longer needed)?  Does all the new
fiemap code still have the ability for me to request that the cp code
specifically look for large blocks of zero in the source, rather than
trusting the fiemap, so that I can create a copy that is more sparse
than the original?  Does that also need a tunable; and if so, should we
try to combine it into this tunable or is it orthogonal?

-- 
Eric Blake   address@hidden    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

signature.asc
Description: OpenPGP digital signature