Re: Bug #33255 (Windows cannot open files with non-ascii names)

[Ben Pfaff]

On Fri, Oct 09, 2015 at 08:02:43AM -0700, Ben Pfaff wrote:
> On Fri, Oct 09, 2015 at 07:29:12AM +0200, John Darrington wrote:
> > On Thu, Oct 08, 2015 at 09:29:35PM -0700, Ben Pfaff wrote:
> >      On Thu, Oct 08, 2015 at 06:47:05PM +0200, John Darrington wrote:
> >      > In addition to the changes I've submitted for review, I think the 
> > resolution to this 
> >      > problem involves work in src/data/make-file.c
> >      > 
> >      > The issue is, this module contains a lot of calls to posix file 
> > functions, such as 
> >      > open, rename, unlink etc.
> >      > 
> >      > These work in w32 so long as there are no non-ascii characters in 
> > the filenames.
> >      > To get other characters to work, we have to do two things:
> >      > 
> >      > 1. Convert the filenames to UTF-16LE
> >      > AND
> >      > 2. replace the function calls by their WideChar equivalents.
> >      > 
> >      > Step 2  however has caveats:
> >      > 
> >      > open --> _wopen is no problem.
> >      > unlink --> _wunlink is no problem.
> >      > rename --> _wrename will not work, because windows rename refuses to 
> > overwrite an existing file.  We will 
> >      >      have to use the native Windows function MoveFileW
> >      > 
> >      > Not sure about stat.
> >      > 
> >      > 
> >      > It may end up being easier to have a platform specific 
> > implementation of make-file.c
> >      > 
> >      > What do you think?
> >      
> >      Can this be fixed in Gnulib somehow?
> > 
> > It probably can be.  The question one has to ask is does it make sense to 
> > do so:
> > 
> > The awkward issue is the _wrename function.   The POSIX rename silently 
> > overwrites
> > the target.  The Windows rename (and presumably the corresponding _wrename) 
> > however 
> > fails with an error if the target exists.
> > 
> > Now some years ago, we (you actually!) contributed a replacement rename to 
> > Gnulib 
> > such that it behaves per POSIX on Windows.  I think it is perfectly 
> > possible to 
> > write a replacement _wrename for w32 which behaves in a more POSIX like way,
> > however then it will not behave how Microsoft's documentation says.
> > 
> > Perhaps it might make sense to have a "widechar" version of rename in 
> > Gnulib, but
> > call it something other than _wrename.
> 
> I guess I was assuming that we'd provide a replacement rename(), or I
> guess enhance the existing rename() replacement, so that it accepted
> UTF-8 file names.  It could be implemented in terms of MoveFileW or
> whatever is appropriate.

Actually, maybe the solution is to use g_rename().  The implementation
looks correct to me:

/**
 * g_rename:
 * @oldfilename: a pathname in the GLib file name encoding (UTF-8 on Windows)
 * @newfilename: a pathname in the GLib file name encoding
 *
 * A wrapper for the POSIX rename() function. The rename() function 
 * renames a file, moving it between directories if required.
 * 
 * See your C library manual for more details about how rename() works
 * on your system. It is not possible in general on Windows to rename
 * a file that is open to some process.
 *
 * Returns: 0 if the renaming succeeded, -1 if an error occurred
 * 
 * Since: 2.6
 */
int
g_rename (const gchar *oldfilename,
          const gchar *newfilename)
{
#ifdef G_OS_WIN32
  wchar_t *woldfilename = g_utf8_to_utf16 (oldfilename, -1, NULL, NULL, NULL);
  wchar_t *wnewfilename;
  int retval;
  int save_errno = 0;

  if (woldfilename == NULL)
    {
      errno = EINVAL;
      return -1;
    }

  wnewfilename = g_utf8_to_utf16 (newfilename, -1, NULL, NULL, NULL);

  if (wnewfilename == NULL)
    {
      g_free (woldfilename);
      errno = EINVAL;
      return -1;
    }

  if (MoveFileExW (woldfilename, wnewfilename, MOVEFILE_REPLACE_EXISTING))
    retval = 0;
  else
    {
      retval = -1;
      switch (GetLastError ())
        {
#define CASE(a,b) case ERROR_##a: save_errno = b; break
          CASE (FILE_NOT_FOUND, ENOENT);
          CASE (PATH_NOT_FOUND, ENOENT);
          CASE (ACCESS_DENIED, EACCES);
          CASE (NOT_SAME_DEVICE, EXDEV);
          CASE (LOCK_VIOLATION, EACCES);
          CASE (SHARING_VIOLATION, EACCES);
          CASE (FILE_EXISTS, EEXIST);
          CASE (ALREADY_EXISTS, EEXIST);
#undef CASE
        default: save_errno = EIO;
        }
    }

  g_free (woldfilename);
  g_free (wnewfilename);
    
  errno = save_errno;
  return retval;
#else
  return rename (oldfilename, newfilename);
#endif
}