Re: [rdiff-backup-users] Native windows build - issue with long path names

[Andrew Ferguson]

Well, here's the answer:

http://msdn.microsoft.com/en-us/library/aa365247.aspx

Some thoughts:

- I believe longname.py is designed just for handling individual  
filenames longer than 255 characters, but I could be wrong.
- Unix systems seem to have a PATH_MAX of 1024 or 4096 ... Windows is  
less than 260, unless you use the Windows API (as opposed to ANSI C,  
like Python uses). This is why you have run into this bug, but there  
have been few (no?) reports of problems before.

So, the next step would be to work out whether longname.py can handle  
situations in which the path name grows longer than PATH_MAX on Unix.  
If so, the logic can be built-on for Windows, if not, rdiff-backup  
will need work on that front.

Then, a patch would need to be written which basically moves all file  
I/O to the Windows API, prepends the "\\?\" (or "\\?\UNC\"), and  
(perhaps) translates to Unicode. Yech. I think this is a star example  
of the stupidity of the Windows kernel & API -- that magic prepend  
business is ridiculous. :-/

some other references:
http://msmvps.com/blogs/access/archive/2008/09/12/maximum-path-length-is-248-characters.aspx
http://msdn.microsoft.com/en-us/library/aa363858(VS.85).aspx


Andrew



On Oct 11, 2008, at 10:31 PM, Josh Nisly wrote:

> I looked into this a bit today. Windows does throw a different  
> exception on long file names than unix (a WindowsError (derived from  
> OSError, derived from EnvironmentError), with the errno set to 206,  
> which doesn't map to anything valid on unix.) However, when I added  
> a check for this to the places that already check for an  
> ENAMETOOLONG exception, it still failed.
>
> JoshN
>
> Ryan How wrote:
> > Just to add, I should point out it is a fatal error in that the  
> > whole backup fails at that point. It isn't just those files that  
> > don't get backed up.
> >
> > Ryan How wrote:
> > > Hi,
> > >
> > > I did a few searches, but I couldn't find any issues discussed  
> > > about long path names in the native windows build.
> > >
> > > I have been having issues with the 255 char limit (or round  
> > > abouts) in pathnames, but I am unsure as to why because I using  
> > > the native windows build (which shouldn't have this limit?). The  
> > > error originates in os.pyc (which, I have no python experience,  
> > > but isn't an rdiff-backup source file?). I get the same error when  
> > > using dos paths (ie. c:\blah) and UNC paths (which otherwise seem  
> > > to work well with the native windows build, yay). I could  
> > > understand the issue with the dos paths, but not with the UNC  
> > > paths. I have a feeling this may actually be a python issue?
> > >
> > > Error is below.
> > >
> > > Cheers,
> > >
> > > Ryan
> > >
> > >
> > > Processing changed file Documents and Settings/ryan/Local Settings/ 
> > > Temp/test_dir_980382089/deep1/deep1_1/deep1_1_1/deep1_1_1_1/ 
> > > deep1_1_1_1_1/deep1_1_1_1_1_1/deep1_1_1_1_1_1_1/deep1_1_1_1_1_1_1_1
> > > Exception '[Error 206] The filename or extension is too long: '\\\ 
> > > \127.0.0.1\\c$\\Documents and Settings\\ryan\\Local Settings\\Temp\ 
> > > \test_dir_2042584358/rdiff-backup-data/increments/Documents and  
> > > Settings/ryan/Local Settings/Temp/test_dir_980382089/deep1/deep1_1/ 
> > > deep1_1_1/deep1_1_1_1/deep1_1_1_1_1/deep1_1_1_1_1_1/ 
> > > deep1_1_1_1_1_1_1'' raised of class '<type  
> > > 'exceptions.WindowsError'>':
> > > File "rdiff_backup\Main.pyc", line 302, in error_check_Main
> > > File "rdiff_backup\Main.pyc", line 322, in Main
> > > File "rdiff_backup\Main.pyc", line 278, in take_action
> > > File "rdiff_backup\Main.pyc", line 344, in Backup
> > > File "rdiff_backup\backup.pyc", line 38, in Mirror
> > > File "rdiff_backup\backup.pyc", line 232, in patch
> > > File "rdiff_backup\rorpiter.pyc", line 284, in __call__
> > > File "rdiff_backup\backup.pyc", line 622, in start_process
> > > File "rdiff_backup\longname.pyc", line 203, in get_mirror_inc_rps
> > > File "rdiff_backup\longname.pyc", line 189, in find_inc_pair
> > > File "rdiff_backup\longname.pyc", line 128, in check_new_index
> > > File "rdiff_backup\longname.pyc", line 111, in wrap_call
> > > File "rdiff_backup\longname.pyc", line 122, in make_parent
> > > File "rdiff_backup\rpath.pyc", line 964, in makedirs
> > > File "os.pyc", line 171, in makedirs
> > >
> > > Traceback (most recent call last):
> > > File "rdiff-backup", line 23, in <module>
> > > File "rdiff_backup\Main.pyc", line 302, in error_check_Main
> > > File "rdiff_backup\Main.pyc", line 322, in Main
> > > File "rdiff_backup\Main.pyc", line 278, in take_action
> > > File "rdiff_backup\Main.pyc", line 344, in Backup
> > > File "rdiff_backup\backup.pyc", line 38, in Mirror
> > > File "rdiff_backup\backup.pyc", line 232, in patch
> > > File "rdiff_backup\rorpiter.pyc", line 284, in __call__
> > > File "rdiff_backup\backup.pyc", line 622, in start_process
> > > File "rdiff_backup\longname.pyc", line 203, in get_mirror_inc_rps
> > > File "rdiff_backup\longname.pyc", line 189, in find_inc_pair
> > > File "rdiff_backup\longname.pyc", line 128, in check_new_index
> > > File "rdiff_backup\longname.pyc", line 111, in wrap_call
> > > File "rdiff_backup\longname.pyc", line 122, in make_parent
> > > File "rdiff_backup\rpath.pyc", line 964, in makedirs
> > > File "os.pyc", line 171, in makedirs
> > > WindowsError: [Error 206] The filename or extension is too long: '\ 
> > > \\\127.0.0.1\\c$\\Documents and Settings\\ryan\\Local Settings\ 
> > > \Temp\\test_dir_2042584358/rdiff-backup-data/increments/Documents  
> > > and Settings/ryan/Local Settings/Temp/test_dir_980382089/deep1/ 
> > > deep1_1/deep1_1_1/deep1_1_1_1/deep1_1_1_1_1/deep1_1_1_1_1_1/ 
> > > deep1_1_1_1_1_1_1'
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > rdiff-backup-users mailing list at address@hidden
> > > http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
> > > Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
> >
> >
> > _______________________________________________
> > rdiff-backup-users mailing list at address@hidden
> > http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
> > Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
>
>
>
> _______________________________________________
> rdiff-backup-users mailing list at address@hidden
> http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
> Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki