Re: [Savannah-hackers-public] disk-usage monitoring on vcs.sv.gnu.org

[Karl Berry]

    Do you mean each repository directory and other 'important'
    directories on the disk, or literally every directory on the disk
    separately (as --separate-dirs does)?

It depends.  If our goal is to determine, at a single point in time,
which projects are consuming the most disk space, then sure, we'd want
to add up the separate subdirs.  I don't see any particular need to run
that from cron -- we could just run the script by hand when the question
arises.

On the other hand, if we're faced with a situation where the disk is
more-or-less suddenly (say) 95+% full, my immediate question would be
which directories have increased consumption the most lately, rather
than which are, overall, consuming the mos.  That's easy to find by
comparing a previous du -S to a current du -S.  I'll append a script I
wrote years ago for that purpose.  (The defaults are for one of my
machines, should be changed for Savannah's purposes.)

In this scenario, I would want to know every single directory, because
the sudden increase could be anywhere.  If directories were coalesced,
then all we would know is "datamash is suddenly taking up 10gb more
space" but we'd have to du the individual directories to learn the
specifics anyway.  So we might as well use du -S in the first place.

Thus, running a du -S weekly or so gives a baseline against which the
current status can be measured.

FWIW ...

Karl

-------------------------------------------------------
#!/usr/bin/env perl
# $Id: cmp-dus,v 1.4 2009/08/24 00:12:26 karl Exp $
# Compare du -S output, current vs. prev.  Public domain.  karl, 16oct04.

exit (&main ());

sub main {
  if (@ARGV == 0) {
    $ARGV[0] = "/u/dus";
    $ARGV[1] = "/backup/u/dus.backup";
  } elsif (@ARGV == 1) {
    $ARGV[0] = "/$ARGV[0]/dus";
    $ARGV[1] = "/backup/$ARGV[0].backup";
  }
  my (%u1) = &read_du ($ARGV[0]);
  my (%u2) = &read_du ($ARGV[1]);
  
  my %diff;
  for my $k (keys %u2) {
    $diff{$k} = $u2{$k} - $u1{$k};
    delete $u1{$k};
  }
  
  # stuff only in u1:
  for my $k (keys %u1) {
    $diff{$k} = -$u1{$k};
  }
  
  # sort diffs.
  for my $k (sort { $diff{$b} <=> $diff{$a} } keys %diff) {
    printf "%10d %s\n", $diff{$k}, $k;
  }
  
  return 0;
}

sub read_du {
  ($FILE) = @_;
  my %ret;
  
  open (FILE) || die "open($FILE) failed: $!";
  while (<FILE>) {
    my ($usage,$dirname) = split (" ", $_);
    $ret{$dirname} = $usage;
  }
  close (FILE) || warn "close($FILE) failed: $!";
  
  return %ret;
}
-------------------------------------------------------