Re: [Bug-ddrescue] ddrescue 10x slow under osx

[Florian Sedivy]

Hola Antonio!

In my experience and testing on OS X, accessing the raw character devices /dev/rdisk… is always preferable. Also the transfer speed can be further enhanced by setting a bigger Copy Block Size. A size of 512KiB (ddrescue -c 1Ki) gave me the best results in most cases. 

And: OS X raw character devices DO have a defined size, so they can be easily used even in the first run. (At least in this point the notes about raw devices in the existing documentation for ddrescue do not apply to OS X.)

I don't think this is a bug in ddrescue, because other utilities like dd or cat exhibit the same behavior on OS X. 

Accessing a /dev/disk… block device gives a rather slow speed, independent of the Copy Block Size used. 

The reading speed of a /dev/rdisk… raw character device on the other hand depends a lot on the Copy Block Size chosen: 

• 512 Byte (ddrescue -c 1, default in dd) is the slowest. 
• Setting it to 4096 Byte (ddrescue -c 8, dd bs=4K) gives the same slow speed as accessing /dev/disk…
• ddrecue's default of 128 sectors (= 64KiB, ddrescue -c 128, dd bs=64K) brings fairly good results.
• Multiplying that further (up to ddrescue -c 1Ki / dd bs=512K) brings maximum speed (mostly 8-12 times faster than /dev/disk…)
• Rising above that did not increase transfer speed any further in my testing; sometimes it even decreased. 

Those are the results of my own measurements, your results may vary depending on the media and IO hardware used. 

Maybe if some other users would share their experience, we could gain a better picture of the topic. 

cat by the way does not let you set a copy block size, but seems to somehow choose one that is optimal. 

cat /dev/disk... >> /dev/null gives the same speed like the other tools when using the block device

cat /dev/rdisk… >> /dev/null seems to give the maximum speed for the raw device

Given that - judging from the participants of the mailing list - approximately half of ddrescue's users are on OS X, my first paragraph might be a vital clue for many. The big question is: where would be the right spot for this information, if you want to keep even the documentation strictly OS-agnostic? 

Greetings, 

Florian

***********

P.S.: From a programming perspective, I see that there is not a lot you could do. Anyway, some considerations:

Obviously the OS X kernel's filesystem access code does not use the block devices at /dev/disk… but some more direct access. The address@hidden group might know more. You (or I) could ask there and maybe some method of accessing a disk directly, that was specified by a /dev/disk… descriptor, exists. Then (if that method is generic enough) it could be inserted as an alternative to the O_DIRECT flag for platforms where that is unsupported. A lot of ifs, I know …

Something else: cat is quite a generic command, and still it manages to somehow select the optimal Copy Block Size for a raw character device on OS X. If the relevant code is not platform specific, it might contain some nice ideas. 

Am 15.12.2012 um 20:05 schrieb Antonio Diaz Diaz:

Hello Garegin, (thanks Florian).

Garegin Asatryan wrote:

Try using /dev/rdiskX as source instead of /dev/diskX. Does that speed

things up?

yes. it fixed the problem. a patch should be made to warn the user, when

running under osx. should buffered i/o cause such huge slowdowns, or is the

problem a bottleneck in the program itself.

Ddrescue's manual already says this:

"Using direct disc access, or reading from a raw device, may be slower or faster than normal cached reading depending on your OS and hardware."

I do not find practical to include concrete warnings for every version of every OS.

I don't know of any bottleneck in ddrescue itself but, if there is one, OS-agnostic patches fixing it are welcome.


Regards,
Antonio.