Re: Feature request for md5sum

[Pádraig Brady]

On 19/02/10 17:55, Scott Nichol wrote:
> I use md5sum a lot, often on network-filesystem-accessed files so that
> readers can be sure they are reading files after writers have completed.
> The performance of md5sum is generally acceptable, but for large
> (gigabyte+) files even on a fast (1 Gbps) LAN, the performance would be
> considerably better if I could specify on the command line a buffer size
> to use.  I imagine the md5sum code using this value in a setvbuf call.
> I use this technique (setting larger buffer sizes for network-accessed
> files) frequently and know md5sum and other utilities would benefit from
> it.

Yes they would benefit, however it would be nice
not to push this complexity into the tools.

Note the buffer size has changed in v8.1 from 4KiB to 32KiB,
which might help.

Also since v7.5 there is a stdbuf utility available
which can be used to change the buffer size for stdin at least:
  stdbuf -i1M md5sum ...
Note there were issues with glibc when I was testing that,
so don't be surprised if the setting ignored.

Note the linux nfs client should be doing adaptive
read ahead to abstract you from this latency issue.
Check the rsize nfs setting as that may limit the
size of read ahead done.

cheers,
Pádraig.