Re: [Bug-ddrescue] ddrescue 10x slow under osx

[Florian Sedivy]

Hola Antonio, 

some additions.

Am 20.12.2012 um 19:11 schrieb Florian Sedivy:

In fact its is BSD's cat. I had a look and the relevant code is:

		if (fstat(wfd, &sbuf))
			err(1, "stdout");
		bsize = MAX(sbuf.st_blksize, BUFSIZ); 

	later:	read(rfd, buf, bsize)

So cat takes st_blksize from stat() and the value of the BUFSIZ macro and uses the bigger one. (cat specifically stats the output 

Ok, so I had another look at different sources and the way IO-Blocksize for cat is determined.

OpenBSD: the above, MAX (stat.st_blksize, BUFSIZ)

different code, same effect, but with the option to set your own buffer size (-B size)

FreeBSD: for regular files stat.st_blksize is ignored, buffer is set according to RAM conditions!

for special files it is MAX (stat.st_blksize, sysconf(_SC_PAGESIZE))

OS X:  hardcoded minimum, MAX (stat.st_blksize, 1024)

As you suspected, GNU coreutils do things a little bit more complicated:

• cat uses MAX ( io_blksize (stat(infile)), io_blksize (stat(outfile)) ) 
• io_blksize() from ioblksize.h returns MAX ( 64KiB, ST_BLKSIZE (stat) )
• the macro ST_BLKSIZE from stat-size.h essentially returns stat.st_blksize in wide limits (0  <  blksize  <  (size_t)/8) and falls back to the disk device block size or 4096

Notably the minimum io_blksize in coreutils (used by cat, copy and split) is 64KiB, which is also ddrescue's default.

For ddrescue the best approach for an adaptive default would probably be to check for the input file's stat.st_blksize and limit that in both directions. Lower limit and fallback value should be the current default of 64KiB. The upper limit does not need to be more than some MiB in my opinion, to make sure ddrescue can still provide regular status updates. As this would only be an (automated) default, users could always override it in both directions with -c. The only purpose is to give a good initial guess for optimum speed in most cases, without having to experiment with the media. 

It would be a small change (faster to change in the code than in the documentation probably), but in my opinion a valuable addition to the already fantastic and incredibly well-kept ddrescue!

Greetings, 

Florian