Re: Bug#467378: coreutils: Please include a program to truncate files

[Jim Meyering]

Russell Coker <address@hidden> wrote:
> If I have a file that is 2G in size but wish to discard the last 1G of data
> then there seems to be no program available to do this.
>
> I think it would be ideal to have a program as part of coreutils that allows
> you to resize a file.  If the new length is longer than the old length then
> it would either write zeros to the end or extend the file (with a hole)
> via the truncate() system call according to the wish of the user.  If the new
> length is shorter then it would just call truncate().
>
> I would be happy to contribute the code for this.  This will require some
> discussion with upstream, but it seemed best to start the discussion here.

That would be nice.

If the mythical miscutils project were more concrete,
I'd say this tool belongs there.

I'm not 100% sure it should be written in C, especially
since I wrote it in Perl:

    $ truncate --help
    Usage: truncate [OPTIONS] FILE ...

    Modify FILE(s) in place, removing part from the end of each of them.
    As a safety measure, if you want to remove more than 10% of a FILE,
    then you must specify the --force option.

    OPTIONS:

       --bytes=-N  remove N bytes from the end of each FILE
       --bytes=N   truncate each FILE to have size N
       --force     don't fail when asked to remove too much
       --help      display this help and exit
       --version   output version information and exit

I used --bytes= to align with head and tail.
If I were to do something official with this code,
I would consider adding aliases, e.g.,

   --from-end=N  remove N bytes from the end of each FILE
   --to-size=N   truncate each FILE to have size N

Russell, if you want to write a C version and do the usual
complete job (update coreutils.texi, NEWS, etc. and add tests),
I'd probably go for it.  The only reason not to would be if
miscutils is now viable, and even then...
One advantage of using C/coreutils/gnulib
is that you'd get k, M, G, ... byte-count suffixes for free.

Here's the script:

#!/usr/bin/perl -w
# Truncate file(s) in place by removing some bytes from the end.

# Copyright (C) 2002, 2007-2008 Free Software Foundation, Inc.

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.

# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.

# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.

# Written by Jim Meyering

use strict;
use Getopt::Long;
use File::stat;

(my $VERSION = '$Revision: 1.13 $ ') =~ tr/[0-9].//cd;
(my $ME = $0) =~ s|.*/||;

# use File::Coda; # http://meyering.net/code/Coda/
END {
  defined fileno STDOUT or return;
  close STDOUT and return;
  warn "$ME: failed to close standard output: $!\n";
  $? ||= 1;
}

my $debug = 0;

# The maximum percentage size decrease.
# If you try to truncate more than this amount, this program
# will fail, unless you override with --force.  But see below...
my $max_percent_size_decrease = 10;

# Even if the above percentage is exceeded, don't fail
# if the number of bytes is smaller than this number.
my $min_bytes_to_fail = 10;

sub usage ($)
{
  my ($exit_code) = @_;
  my $STREAM = ($exit_code == 0 ? *STDOUT : *STDERR);
  if ($exit_code != 0)
    {
      print $STREAM "Try `$ME --help' for more information.\n";
    }
  else
    {
      my $p = $max_percent_size_decrease;
      print $STREAM <<EOF;
Usage: $ME [OPTIONS] FILE ...

Modify FILE(s) in place, removing part from the end of each of them.
As a safety measure, if you want to remove more than $p% of a FILE,
then you must specify the --force option.

OPTIONS:

   --bytes=-N  remove N bytes from the end of each FILE
   --bytes=N   truncate each FILE to have size N
   --force     don't fail when asked to remove too much
   --help      display this help and exit
   --version   output version information and exit

EOF
    } #'
  exit $exit_code;
}

{
  my $n_bytes;
  my $force;

  GetOptions
    (
     'bytes=i' => \$n_bytes,
     force => \$force,
     debug => \$debug,
     help => sub { usage 0 },
     version => sub { print "$ME version $VERSION\n"; exit },
    ) or usage 1;

  # Make sure we have at least one non-option argument.
  @ARGV < 1
    and (warn "$ME: missing FILE argument\n"), usage 1;

  defined $n_bytes
    or (warn "$ME: you must specify --bytes=N\n"), usage 1;

  my $fail = 0;
  foreach my $arg (@ARGV)
    {
      open F, '+<', $arg
        or (warn "$ME: failed to open $arg for read/write\n"), $fail = 1, next;

      my $st = stat ($arg)
        or (warn "$ME: failed to stat $arg\n"), $fail = 1, next;

      my $size = $st->size;
      my $new_size;
      if ($n_bytes < 0)
        {
          $new_size = $size + $n_bytes;
          $new_size < 0
            and $new_size = 0;
        }
      else
        {
          $size <= $n_bytes
            and (warn "$ME: warning: specified size ($n_bytes) is the same as,"
                 . " or larger than the size of $arg\n"), next;
          $new_size = $n_bytes;
        }

      (# Don't warn if the user said --force.
       $force
       # Don't warn if we're removing some small number of bytes.
       # I.e. don't warn if we're removing 20% of a 5-byte file.
       || $size - $new_size < $min_bytes_to_fail
       # Don't warn if we are removing $max_percent_size_decrease% or less.
       || 100 * ($size - $new_size) <= $size * $max_percent_size_decrease)
        or (warn "$ME: ERROR: truncating $arg to length $new_size"
            . " would remove more than $max_percent_size_decrease%;\n"
            . "  override with --force\n"),
              $fail = 1, next;

      truncate F, $new_size
        or (warn "$ME: failed to truncate $arg to length $new_size\n"),
          $fail = 1, next;

      close F
        or (warn "$ME: failed to close $arg\n"),
          $fail = 1, next;
    }

  exit $fail;
}