Re: [rdiff-backup-users] Encountering the NFS "Directory not empty" error

[Bernd Schubert]

Hello Ben,

> I had considered the possibility that rdiff-backup was failing to
> close files, but I spent some time looking through the code and
> couldn't find any loose files.  Also from the reports I've gotten it
> seems this error occurs inconsistently, and at different places, even
> though rdiff-backup is single-threaded and purely deterministic.
>
> But your paragraph explains the mechanism well, and suggests that it's
> an rdiff-backup problem after all.

I already checked my theory some time ago, by just putting a long sleep before 
the rmdir exception. Then I looked into the directory and it showed .nfs 
files. Furthermore /proc/{PID of rdiff-backup}/fd showed open filedescriptors 
to those files.

>
> > A few month ago I already tried to fix this, but did it wrong (due
> > to my lack of Python knowledge), unfortutely my lack of time (need
> > to finish my Ph.D. thesis as soon as possible, but there are still
> > unsolved problems) prevented further me to look deeper into it :( I
> > will really do another attempt this or the next weekend.
>
> From my perspective, the hard part seems to be replicating the problem
> consistently, and figuring out exactly which file(s) are not getting
> closed.  If you can do this (which should be doable with no Python
> knowledge), then it may be easy for me or someone else to fix it.

Well, replicating is very easy, I think I can do this within seconds, I only 
need one file in one dir. So I think its also easy to figure out which file 
it is. From my point of view its difficult to find out the corresponding 
open() in rdiff-backup. I already tried a python-debugger, but it always 
fails with another exception (I believe to remember its an open() to a 
non-existent file), I also still do not understand why it doesn't rise this 
exception without a debugger. If you are also interested in this issue, I can 
tell you tomorrow or on Tuesday the exact problem. 
Another possibility would be do monitor all open() and close() calls, but I 
think there are rather many of them and all of them would need a print or 
somethink link that. 

Another point I also still don't understand is how the deleted files reappear 
in the directory on the next rdiff-backup run. In principle the .nfs files 
are immediately deleted after rdiff-backup has rised its exception and closes 
itself (that's why one usually can't see the .nfs files). In principle I 
would expect rdiff-backup to succeed on the next run, but on the next run 
those already deleted files reappear again :(

Sorry, today I again had no time to further care about it (its time to go to 
sleep now), but we have a public holiday on Tuesday and I will try to look 
into it agai tomorrow evening or on Tuesday.

>
> Good luck with your thesis :-)

Thanks, I really need it (we need to find a numerical workaround for a 
mathematical singularity and now its seems only a small step is missing).

Cheers,
        Bernd

-- 
Bernd Schubert
PCI / Theoretische Chemie
Universität Heidelberg
INF 229
69120 Heidelberg