Re: [patch #6797] shred option to use internal RNG

[Pádraig Brady]

Pádraig Brady wrote:
> Jim Meyering wrote:
>> Thanks for doing that.
>> At first glance, you got it all.
>> Except maybe the texinfo documentation...
>> Yep.  There's a mention of shred in the "Random sources" section.
>> So now we'd have to say how shred's --random-source works
>> and that its default is different from those of shuf and sort...
>>
>> Have you considered keeping the semantics of --random-source
>> the same for all three programs?
> 
> I did, but was conservative and only steered the bulk
> random generator (shred) away from /dev/urandom by default.
> 
>> Things that ease the documentation burden by keeping inter-tool
>> differences to a minimum are usually also good from usability
>> and maintainability perspectives.

How about the attached.

cheers,
Pádraig.

>From 1e775072c2181679a088794dc872dc78a41a7606 Mon Sep 17 00:00:00 2001
From: =?utf-8?q?P=C3=A1draig=20Brady?= <address@hidden>
Date: Mon, 6 Apr 2009 08:42:15 +0100
Subject: [PATCH] shred,sort,shuf: don't use /dev/urandom by default

Suggestion from Steven Schveighoffer at:
http://savannah.gnu.org/patch/?6797
to greatly speed up the random passes done by shred.
* gl/lib/randread.c: Default to using the internal
pseudorandom generator, rather than reading /dev/urandom
* src/shred.c (usage): remove mention of /dev/urandom
* src/shuf.c (usage); ditto
* src/sort.c (usage): ditto
* doc/coreutils.text: Document the new behaviour
for aquiring random data.
---
 NEWS               |    7 +++++++
 THANKS             |    1 +
 doc/coreutils.texi |   22 +++++++++-------------
 gl/lib/randread.c  |   24 +++++++-----------------
 src/shred.c        |    2 +-
 src/shuf.c         |    2 +-
 src/sort.c         |    2 +-
 7 files changed, 27 insertions(+), 33 deletions(-)

diff --git a/NEWS b/NEWS
index 7fdbd08..981447d 100644
--- a/NEWS
+++ b/NEWS
@@ -7,6 +7,13 @@ GNU coreutils NEWS                                    -*- 
outline -*-
   ls now aligns output correctly in the presence of abbreviated month
   names from the locale database that have differing widths.
 
+** Changes in behavior
+
+  shred, sort, shuf: now use an internal pseudorandom generator by default.
+  This is mainly noticable in shred where the 3 random passes it does by
+  default should proceed at the speed of the disk.  Previously /dev/urandom
+  was used if available, which is relatively slow on GNU/Linux systems.
+
 * Noteworthy changes in release 7.2 (2009-03-31) [stable]
 
 ** New features
diff --git a/THANKS b/THANKS
index 11f43ac..6a918a4 100644
--- a/THANKS
+++ b/THANKS
@@ -525,6 +525,7 @@ Steve McIntyre                      address@hidden
 Steve Ward                          address@hidden
 Steven G. Johnson                   address@hidden
 Steven Mocking                      address@hidden
+Steven Schveighoffer                address@hidden
 Steven P Watson                     address@hidden
 Stuart Kemp                         address@hidden
 Stuart Shelton                      address@hidden
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 70effa1..045fbd1 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -1139,12 +1139,19 @@ sometimes need random data to do their work.  For 
example, @samp{sort
 -R} must choose a hash function at random, and it needs random data to
 make this selection.
 
-Normally these commands use the device file @file{/dev/urandom} as the
+By default these commands use an internal pseudorandom generator
+initialized by a small amount of entropy, but can be directed to use
+an external source with the @address@hidden option.
+An error is reported if @var{file} does not contain enough bytes.
+
+For example, the device file @file{/dev/urandom} could be used as the
 source of random data.  Typically, this device gathers environmental
 noise from device drivers and other sources into an entropy pool, and
 uses the pool to generate random bits.  If the pool is short of data,
 the device reuses the internal pool to produce more bits, using a
-cryptographically secure pseudorandom number generator.
+cryptographically secure pseudorandom number generator.  But be aware
+that this device is not designed for bulk random data generation
+and is relatively slow.
 
 @file{/dev/urandom} suffices for most practical uses, but applications
 requiring high-value or long-term protection of private data may
@@ -1152,21 +1159,10 @@ require an alternate data source like 
@file{/dev/random} or
 @file{/dev/arandom}.  The set of available sources depends on your
 operating system.
 
-To use such a source, specify the @address@hidden
-option, e.g., @samp{shuf --random-source=/dev/random}.  The contents
-of @var{file} should be as random as possible.  An error is reported
-if @var{file} does not contain enough bytes to randomize the input
-adequately.
-
 To reproduce the results of an earlier invocation of a command, you
 can save some random data into a file and then use that file as the
 random source in earlier and later invocations of the command.
 
-Some old-fashioned or stripped-down operating systems lack support for
address@hidden/dev/urandom}.  On these systems commands like @command{shuf}
-by default fall back on an internal pseudorandom generator initialized
-by a small amount of entropy.
-
 @node Target directory
 @section Target directory
 
diff --git a/gl/lib/randread.c b/gl/lib/randread.c
index b81a451..798d4e0 100644
--- a/gl/lib/randread.c
+++ b/gl/lib/randread.c
@@ -50,10 +50,6 @@
 # define ALIGNED_POINTER(ptr, type) ((size_t) (ptr) % alignof (type) == 0)
 #endif
 
-#ifndef DEFAULT_RANDOM_FILE
-# define DEFAULT_RANDOM_FILE "/dev/urandom"
-#endif
-
 /* The maximum buffer size used for reads of random data.  Using the
    value 2 * ISAAC_BYTES makes this the largest power of two that
    would not otherwise cause struct randread_source to grow.  */
@@ -62,10 +58,8 @@
 /* A source of random data for generating random buffers.  */
 struct randread_source
 {
-  /* Stream to read random bytes from.  If null, the behavior is
-     undefined; the current implementation uses ISAAC in this case,
-     but this is for old-fashioned implementations that lack
-     /dev/urandom and callers should not rely on this.  */
+  /* Stream to read random bytes from.  If null, the current
+     implementation uses an internal PRNG (ISAAC).  */
   FILE *source;
 
   /* Function to call, and its argument, if there is an input error or
@@ -147,18 +141,14 @@ randread_new (char const *name, size_t bytes_bound)
     return simple_new (NULL, NULL);
   else
     {
-      char const *file_name = (name ? name : DEFAULT_RANDOM_FILE);
-      FILE *source = fopen_safer (file_name, "rb");
+      FILE *source = NULL;
       struct randread_source *s;
 
-      if (! source)
-       {
-         if (name)
-           return NULL;
-         file_name = NULL;
-       }
+      if (name)
+       if (! (source = fopen_safer (name, "rb")))
+         return NULL;
 
-      s = simple_new (source, file_name);
+      s = simple_new (source, name);
 
       if (source)
        setvbuf (source, s->buf.c, _IOFBF, MIN (sizeof s->buf.c, bytes_bound));
diff --git a/src/shred.c b/src/shred.c
index 6ed4daa..69bc226 100644
--- a/src/shred.c
+++ b/src/shred.c
@@ -167,7 +167,7 @@ Mandatory arguments to long options are mandatory for short 
options too.\n\
       printf (_("\
   -f, --force    change permissions to allow writing if necessary\n\
   -n, --iterations=N  overwrite N times instead of the default (%d)\n\
-      --random-source=FILE  get random bytes from FILE (default 
/dev/urandom)\n\
+      --random-source=FILE  get random bytes from FILE\n\
   -s, --size=N   shred this many bytes (suffixes like K, M, G accepted)\n\
 "), DEFAULT_PASSES);
       fputs (_("\
diff --git a/src/shuf.c b/src/shuf.c
index 977eedc..b221d03 100644
--- a/src/shuf.c
+++ b/src/shuf.c
@@ -62,7 +62,7 @@ Mandatory arguments to long options are mandatory for short 
options too.\n\
   -i, --input-range=LO-HI   treat each number LO through HI as an input line\n\
   -n, --head-count=COUNT    output at most COUNT lines\n\
   -o, --output=FILE         write result to FILE instead of standard output\n\
-      --random-source=FILE  get random bytes from FILE (default 
/dev/urandom)\n\
+      --random-source=FILE  get random bytes from FILE\n\
   -z, --zero-terminated     end lines with 0 byte, not newline\n\
 "), stdout);
       fputs (HELP_OPTION_DESCRIPTION, stdout);
diff --git a/src/sort.c b/src/sort.c
index 5b63a25..2e6ce87 100644
--- a/src/sort.c
+++ b/src/sort.c
@@ -339,7 +339,7 @@ Ordering options:\n\
       fputs (_("\
   -n, --numeric-sort          compare according to string numerical value\n\
   -R, --random-sort           sort by random hash of keys\n\
-      --random-source=FILE    get random bytes from FILE (default 
/dev/urandom)\n\
+      --random-source=FILE    get random bytes from FILE\n\
   -r, --reverse               reverse the result of comparisons\n\
 "), stdout);
       fputs (_("\
-- 
1.5.3.6