Re: [Bug-ddrescue] partially recovered mftshort

[Scott D]

Hi Andy

Technically this is Antonio's ddrescue bug list, and most of your questions are related to my ddrutiltiy software so those maybe should have been asked in the discussion forum for ddrutility on its sourceforge page, but I will answer them here since they were asked here.

On 5/7/2014 11:59 AM, Andy Malcolm wrote:

I made a backup of the logfiles per $Bitmap and followed the multi CD merge instructions from the ddrescue manual on /dev/sda and /dev/sde in the range pertaining to the bitmap and that seemed to work. I was sceptical but it seemed like copying the bitmap from /dev/sda was going to take at least a week so I went with it.

First, the $Bitmap file is not needed for actual file recovery. Ddru_ntfsbitmap only uses it as a tool to help only recover the used parts and not wasting time on unused space. I would not recommend spending any extra time at all on fully recovering it. Second, merging the bitmap file from the target disk will very likely leave inaccurate results. I would not at all trust the domain file created from that. Ddru_ntfsbitmap will normally compensate for the errors in the bitmap file and consider all those areas as "used" disk space to be on the safe side when creating the domain file. When you merge from the target you will have no idea what is good or bad, and will end up with a domain file that will not include areas that it should. Also, you have no idea what data was previously on the target (unless you wrote 0's to the entire drive before you started), and since ddrescue does not write to the target where the errors are in the source, you are introducing possible random data from the merge. I would go back to the backups of the bitmap file and bitmap log file to re-create the domain file. If you did not make a copy of the bitmap file then you would have to delete both the bitmap file and bitmap logfile and run ddru_ntfsbitmap again. If the bitmap file is that damaged then it is likely not worth any retry.

Also, since you had such trouble getting a good copy of mftshort I would make an extra backup of that and its logfile so you don't have to worry about ddru_ntfsbitmap deleting it on a new run.

Then I tried ddru_ntfsbitmap /dev/sda4 domain_logfile

which yielded:
"__mftshort.log
__mftshort

current pos:     3221 MB,  current status: finished
domain size:     16384 B,  in    1 area(s)
    rescued:         0 B,  in    0 area(s)  (  0%)
  non-tried:         0 B,  in    0 area(s)  (  0%)

    errsize:     16384 B,  errors:       1  (100%)
non-trimmed:         0 B,  in    0 area(s)  (  0%)
  non-split:         0 B,  in    0 area(s)  (  0%)
 bad-sector:     16384 B,  in    1 area(s)  (100%)"

so I ran this

sudo ddrescue -i3221225472 -o0 -s16384 -r-1 -R /dev/sda4 __mftshort __mftshort.log

You could have used the "--options" option of ddru_ntfsbitmap to achieve this:

    ddru_ntfsbitmap /dev/sda4 domain_logfile --options "-r-1 -R"

That option is there so you can add whatever ddrescue options you wish for retries, max-errors, and such without having to do a manual partial recovery like you did. Anything you put inside the quotes will be passed directly to the ddrescue command line.

Also, you should really use the "--mftdomain" option for your case since you know the MFT has errors:

    ddru_ntfsbitmap /dev/sda4 domain_logfile --mftdomain mftdomain_logfile

Assuming you did correctly and successfully recover the mftshort, then this option should create a domain file of the $MFT itself. While the bitmap file is not important for actual file recovery, the MFT is most DEFINITELY needed for proper file recovery. If you can create a domain file for the MFT that would be much more productive. You could then run ddrescue using the mftdomain_logfile to see how bad the MFT actually is. If you are going to focus on anything, focus on that first. Then you could move to using the regular domain file from the bitmap if it is usable. The MFT domain file should be created before the regular bitmap domain file, so once ddru_ntfsbitmap starts trying to read the bitmap file you should be able to stop it and have a usable mftdomain domain file.

    ddrescue -f /dev/sda /dev/sde ddrlog.txt -m mftdomain_logfile

As I look at my documentation, I notice that I did not include any good examples of of that. Has been added to the "to-do" list.

Since I've already started a /dev/sda to /dev/sde with ddrescue, is there some way to write these ntfsbitmap retrying results onto /dev/sde and update my original ddrlog.txt, like my 16k mftshort data which I don't think made it from /dev/sda to /dev/sde originally?

Sorry, there is no easy way to write any single files recovered to the target without doing it manually, and I am not going to get into that here. Maybe someday I will add that ability, but it would be very low on my priority list. Even I have to work at it if I want to do it. All I can say is I think you have to use dd instead of ddrescue, as ddrescue does not like the size difference when the source is a small file being written to a large disk, I know something doesn't work right as I have tried in the past (maybe Antonio can answer why that doesn't work).

since /dev/sde is exactly the same size (capacity) and layout as /dev/sda and a partial ddrescue copy of /dev/sda could I just run the ntfsbitmap commend on sde first and then on sda?

In a word, NO! Ddru_ntfsbitmap NEEDS to see the errors to properly adjust the bitmap file to make a safe domain logfile.

Lastly, is a partially recovered $Bitmap at all usefull?

Yes, at least for ddru_ntfsbitmap. See the documentation about how the error parts are treated as used disk space to be safe. If it is still unfinished to the point where ddru_ntfsbitmap isn't done because ddrescue is still reading it, and won't produce a domain file, then you could pass options to ddrescue such as --max-errors or --max-error-rate to get ddrescue to exit normally before finishing and let ddru_ntfsbitmap create the best domain file it can. Again, the domain file will be created "safe", as any area of the bitmap file that was not successfully read will be filled with "FF" to be considered used space so it will be attempted to rescue when using the created domain file.


Scott