bug#29189: 25.3; Dired does not work with binary filenames

[Eli Zaretskii]

> Date: Thu, 16 Nov 2017 18:00:55 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 29189@debbugs.gnu.org, schwab@suse.de
> 
> > From: Allen Li <vianchielfaura@gmail.com>
> > Date: Wed, 15 Nov 2017 22:31:48 -0800
> > Cc: handa@gnu.org, Andreas Schwab <schwab@suse.de>, 29189@debbugs.gnu.org
> > 
> > > diff --git a/lisp/files.el b/lisp/files.el
> > > index b47411f..43198bc 100644
> > > --- a/lisp/files.el
> > > +++ b/lisp/files.el
> > > @@ -6803,10 +6803,13 @@ insert-directory
> > >                             val (get-text-property (point) 
> > > 'dired-filename))
> > >                       (goto-char (next-single-property-change
> > >                                   (point) 'dired-filename nil 
> > > (point-max)))
> > > -                     ;; Force no eol conversion on a file name, so
> > > -                     ;; that CR is preserved.
> > > -                     (decode-coding-region pos (point)
> > > -                                           (if val coding-no-eol coding))
> > > +                      (let ((fn (buffer-substring-no-properties pos 
> > > (point))))
> > > +                        (delete-region pos (point))
> > > +                        (insert
> > > +                        ;; Force no eol conversion on a file name, so
> > > +                        ;; that CR is preserved.
> > > +                        (decode-coding-string (string-make-unibyte fn)
> > > +                                              (if val coding-no-eol 
> > > coding))))
> > >                       (if val
> > >                           (put-text-property pos (point)
> > >                                              'dired-filename t)))))))
> > 
> > This patch works for me.
> 
> Thanks for testing.  I'm still worried that we need to force text to
> be unibyte in order for the decoding to work.  So I'd like to dig into
> the code to understand why, and maybe try to fix it if I find some
> problems there.  If I succeed, the result will work faster, because
> the above patch is less efficient that decode-coding-region.  Let me
> look into this and get back to you in a few days.

I found that the alternative patch below solves the original problem
without any changes needed in files.el, and without introducing any
performance hits.  Does anyone see a problem with this proposed patch?
Kenichi?

diff --git a/src/coding.c b/src/coding.c
index d790ad0..eaad0d7 100644
--- a/src/coding.c
+++ b/src/coding.c
@@ -7423,10 +7423,21 @@ decode_coding (struct coding_system *coding)
 
          while (nbytes-- > 0)
            {
-             int c = *src++;
+             int c;
 
-             if (c & 0x80)
-               c = BYTE8_TO_CHAR (c);
+             /* Copy raw bytes in their 2-byte forms as single characters.  */
+             if (CHAR_BYTE8_HEAD_P (*src) && nbytes > 0)
+               {
+                 c = STRING_CHAR_ADVANCE (src);
+                 nbytes--;
+               }
+             else
+               {
+                 c = *src++;
+
+                 if (c & 0x80)
+                   c = BYTE8_TO_CHAR (c);
+               }
              coding->charbuf[coding->charbuf_used++] = c;
            }
          produce_chars (coding, Qnil, 1);