proposal for change cp default copying behavior ...

[Pádraig Brady]

... to do the lightest weight possible copy.
Related to the details below are the replacement of
the current fiemap code (which requires sync to use),
with SEEK_HOLE and SEEK_DATA.

cheers,
Pádraig.


-------- Forwarded Message --------
Subject: Re: Bug#804338: coreutils: provide a clean way for optional global cp 
--reflink=auto
Date: Sat, 7 Nov 2015 16:15:21 +0000
From: Pádraig Brady <address@hidden>
To: Christoph Anton Mitterer <address@hidden>, address@hidden

On 07/11/15 14:34, Christoph Anton Mitterer wrote:
> Package: coreutils
> Version: 8.23-4
> Severity: wishlist
> 
> 
> Hi.
> 
> It would be nice if the package could provide a clean out-of-the-box
> way to get global --reflink=auto behaviour.
> 
> Starting with 8.24, mv --reflink=auto will be the default, but
> for cp it stays at "never", and probably always will, see Pádraig's
> comment here:
> https://unix.stackexchange.com/questions/80351/why-is-cp-reflink-auto-not-the-default-behaviour/152639#152639
> 
> 
> I have no opinion on whether these arguemnts are good or not,
> some may argue they justify not using reflink per default, some
> may say that people choosing filesystems as btrfs typically expect/want
> their drawbacks/advantages.
> 
> 
> Anyway, it should be easily possible to switch the global default.
> 
> Shell aliases are not a solution, they're typically not expanded in
> non-interactive shells (and while at least bash allows to do so, I'd
> say this is rather dangerous as it potentially breaks countless
> scripts) and as such any system scripts, cron jobs (that make e.g.
> backups) or any other program that directly invokes cp wouldn't benefit
> from it.
> 
> Creating a wrapper shell script and dpkg-diverting the whole story
> doesn't seem proper either, as the wrapper would require additional
> shell execution... and why should people who want the more "native"
> btrfs behaviour have to suffer from such penalty?
> 
> Maybe the coreutils package could provide a 2nd cp binary with
> changed default and people could have things more easily switched,
> either debconf (probably difficult for coreutils ;) ) or some
> proper end-user friendly documentation on how to dpkg-divert.
> 
> 
> Cheers,
> Chris.
> 
> PS: Oh and having 8.24 in Debian would be nice, either
> PS2: I've CCed Pádraig so he may comment if there are any better ways
> to provide solution for this upstream, or even if upstream's view
> may change on the cp/reflink=auto question.

The reason --reflink=auto is not default for cp is because it
changes the end state of the copy. In this case the main issue is
that later changes to the file may give ENOSPC or result in fragmentation.
Now I do agree that you'll get the same issue if you later fill holes
in the file, or change data on deduplicating file systems/devices, ...
So perhaps we should just consider reflink as an implementation option,
and have a higher level --prealloc or whatever for those who care about that.
I.E. instead have cp to do the lightest weight copy possible,
and have --prealloc etc. change that for the edge cases that care
(with the caveat that existing systems may already be depending
 on the current non CoW default behavior).

There is work currently in the kernel for providing vfs_copy_file_range()
which will copy at the lowest level possible. That will do in prefered order;
server copy offload, SCSI COPY, reflink on BTRFS (or XFS etc. when available),
or splice at the VFS level. If cp calls into that by default then
you get your desired behavior by default. I'm looking at that logic
at present, and hope to have it in the next coreutils release,
which would be v9.0 since there are other invasive changes being considered.

cheers,
Pádraig.