Re: [rdiff-backup-users] Re: Fatal Error: Source is not a directory [Scanned]

[Dave]

More info/questions:

I did a compare and the backed up data is not deleted. It is just not visible with ordinary file system tools (such as Nautilus).

I also noticed that I have two current mirror files at the backup destination where I'm having this problem.

That lead me to this FAQ:

1. For some reason rdiff-backup failed while backing up. Now every time it runs it says "regressing destination" and then fails again. What should I do?

   Firstly, this shouldn't happen. If it does, it indicates a corrupted destination directory, a bug in rdiff-backup, or some other serious recurring problem.

   However, here is a workaround that you might want to use, even though it probably won't solve the underlying problem: In the destination's rdiff-backup-data directory, there should be two "current_mirror" files, for instance:

   current_mirror.2003-09-07T16:43:00-07:00.data
   current_mirror.2003-09-08T04:22:01-07:00.data

   Delete the one with the earlier date. Also move the mirror_metadata file with the later date out of the way, because it probably didn't get written correctly because that session was aborted:

   mv mirror_metadata.2003-09-08T04:22:01-07:00.snapshot.gz aborted-metadata.2003-09-08T04:22:01-07:00.snapshot.gz

   The next time rdiff-backup runs it won't try regressing the destination. Metadata will be read from the file system, which may result in some extra files being backed up, but there shouldn't be any data loss.

I tried those steps and I was not very happy with the results.

In the end I deleted the entire backup and started fresh. It took about 3 hours to complete the new backup. Then I ran another backup soon after and it finished in 10 minutes. So now things appear to be back to normal.

I do not know why that problematic backup ran for 9 hours without finishing, and I do not know what "corrupted" my backup. But the only thing that changed was introducing a script (discussed in this thread) to automatically change directories on another separate backup. May be coincidence. I have no idea. I'm just reporting my experiences. Hope it helps.

Dave

On Wed, Apr 22, 2009 at 4:31 PM, Dave <address@hidden> wrote:

I want to follow up on this thread. Something strange happened today (see "Should I kill the rdiff-backup process? Fatal Error: a previous rdiff-backup session is still running").

Here's the full story. I have been running an rdiff-backup to a local HDD hourly. It completes in about 7 minutes and 50-59 seconds like clockwork.

Yesterday I set up a separate rdiff-backup where the destination is another computer on my LAN. I plan to run this just once a day. And I used the strategy discussed in this thread of changing to the path of a symlink and backing up from there.

I ran the first backup using this symlink strategy and it completed fine.

Then, some time later, a normal hourly backup started. (It uses a local destination and backs up up different stuff.) But instead of finishing in 7 minutes and 50-some seconds as it always does, it kept going for 9 hours. I finally killed it.

Next hour, I let it start again, and it also ran past the usual time, so I killed it after about 45 minutes.

I inspected the backup location (local disk) and it appears most of the backed up data has been deleted.

I have no explanation for what happened (I'm a new rdiff-backup user), but the only thing that changed was introducing the symlink strategy discussed in this thread.

I'm not sure what my next step will be, but I thought my experiences would interest the list.

Regards,
Dave

On Tue, Apr 21, 2009 at 8:35 PM, Ty! Boyack <address@hidden> wrote:

I think Jason's suggestion will still work for you.  You'll just have to change directory inside your script.

A small script like:
/
#!/bin/bash

cd /home/user/fileToBackup
rdiff-backup . /backup/directory

/
is certainly valid.  Of course, that can be part of a larger script.  If you are worried about the rest of your script having troubles because you have changed directories, you can follow it with 'cd -' (cd<space><dash>) which in most shells will return you to your last directory -- in essence it will negate the previous 'cd /home/user/fileToBackup' command.

You say that the link changes frequently -- this may be the safest way then.  The only time the link will be resolved is during the 'cd' command.  After that if the link changes while you are doing your backup it won't matter to rdiff-backup.
If you don't cd and tell rdiff-backup to follow the symlink, and the symlink changed during rdiff-backup's run, it might make a mess of things.  I don't know enough of rdiff-backup's internals to know if that would be safe or not, but I think this way would help ensure that you are not going to get in trouble from that.

-Ty!

Dave wrote:

Hi Jason,
That's very helpful info. I can't just cd to the directory because this backup is happening in a script and the actual directory (the one pointed to by the symlink) changes frequently.

My current version of Ubuntu is using Python 2.5.2. I will look into upgrading. (I'm planning on installing Jaunty when it is released.)

Regards,
Dave

On Tue, Apr 21, 2009 at 8:11 PM, Jason Spalding <address@hidden <mailto:address@hidden>> wrote:

   I was going to suggest this but it doesn't work for me on my OS X
   system.
   Source diving lead me to the lstat() function in
   backup_check_dirs, but the
   error reported is for the isdir() function directly below it
   which, for all
   intents and purposes, should operate correctly (it's a python
   call, not
   rdiff's). Perhaps something for the dev team to look at later.

   In the meantime, a workaround like so should suffice:

   cd /home/user/fileToBackup/
   rdiff-backup . /backup/directory

   Which should solve your problem.

   A few notes on looking into it:

   The isdir function is documented like so at docs.python.org

   <http://docs.python.org>:

   os.path.isdir(path)¶
      Return True if path is an existing directory. This follows symbolic
   links, so both islink() and isdir() can be true for the same path.

   However this is for Python version 2.6.2 - my version of OS X is a 2.5
   installation and as far as I am aware Apple have no plans to
   upgrade this in
   the foreseeable future. As such the older version of python may be
   what is
   causing the no-follow on the symlink for directory testing. I
   couldn't find
   the return codes for the lstat function (I've never enjoyed python
   documentation), nor the 2.5 docs (though I could have googled
   further) but
   it would appear that when lstat() is not following symbolic links, it
   returns everything ok - thus falling through to isdir().

   The above cd / rdiff combination worked just fine for me though.

   Cheers

   Jason
   (Not an rdiff-backup developer)

   On 22/4/09 9:27 AM, "Ty! Boyack" <address@hidden

   <mailto:address@hidden>> wrote:

   > With many programs the answer is to simply add a slash to the end of
   > your source directory.  For example if your command was:
   >
   > rdiff-backup /home/user/fileToBackup  /backup/directory
   >
   > then just change it to:
   >
   > rdiff-backup /home/user/filesToBackup/ /backup/directory
   >
   > the name 'fileToBackup' is a symlink, but when you append a slash it
   > forces it to resolve to the destination of the symlink.
   >
   > Not certain it works here, but I'd try that first.
   >
   > -Ty!
   >
   >
   > Dave wrote:
   >> Anyone know if there is a way to backup using a symlink as the
   source?
   >> Thanks
   >>
   >> On Mon, Apr 20, 2009 at 5:41 PM, Dave <address@hidden
   <mailto:address@hidden>

   >> <mailto:address@hidden

   <mailto:address@hidden>>> wrote:
   >>
   >>     I called rdiff-backup with a symlink as the source I wanted to
   >>     backup and I got this error:
   >>
   >>     Fatal Error: Source /home/user/filesToBackup is not a directory
   >>
   >>     Is there a way to backup a symlink like this? Thanks.
   >>
   >>     Dave
   >>
   >>
   >>
   ------------------------------------------------------------------------
   >>
   >> _______________________________________________
   >> rdiff-backup-users mailing list at

   address@hidden <mailto:address@hidden>

   >> http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
   >> Wiki URL:
   http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
   >



------------------------------------------------------------------------

_______________________________________________
rdiff-backup-users mailing list at address@hidden
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

--
-===========================-
 Ty! Boyack
 NREL Unix Network Manager
 address@hidden
 (970) 491-1186
-===========================-