bug#10299: Emacs doesn't handle Unicode characters in keyboard layout on MS Windows

[Joakim Hårsman]

On 16 December 2011 09:13, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Thu, 15 Dec 2011 22:47:17 +0100
>> From: Joakim Hårsman <joakim.harsman@gmail.com>
>> Cc: Jason Rumney <jasonr@gnu.org>, 10299@debbugs.gnu.org, handa@m17n.org
>>
>> > gcc -I. -c -gdwarf-2 -g3  -mtune=pentium4 -O2      -Demacs=1 
>> > -DHAVE_CONFIG_H -I.
>> > ./lib -I../nt/inc -DHAVE_NTGUI=1 -DUSE_CRT_DLL=1 -DPURESIZE=5000000 
>> > -DPURESIZE=5
>> > 000000 -o oo-spd/i386/make-docfile.o make-docfile.c
>> > make-docfile.c:36:20: config.h: No such file or directory
>> > make-docfile.c:79: error: syntax error before "NO_RETURN"
>> > make-docfile.c:79: warning: data definition has no type or storage class
>> > make[8]: *** [oo-spd/i386/make-docfile.o] Error 1
>> > make[8]: Leaving directory `D:/Dev_projects/emacs/trunk/lib-src'
>> > make[7]: *** [bootstrap-gmake] Error 2
>> > make[7]: Leaving directory `D:/Dev_projects/emacs/trunk/nt'
>> >
>> > I'm not really sure what to make of this, config.h is in trunk/src,
>> > but gcc doesn't seem to be looking there from what I can see. Are
>> > there more detailed build instructions available or is nt/INSTALL all
>> > there is?
>> >
>> > All I'm doing is cd:ing into the nt directory, running configure and
>> > then make bootstrap.
>>
>> Never mind, I got the build to work.
>>
>> It turns out I only got the above error the second time I ran "make
>> bootstrap", presumably because some environment variable lingers from
>> the first invocation and messes things up.
>
> That compilation failed because -I../src is missing from the command
> line:
>
>> gcc -I. -c -gdwarf-2 -g3 -mtune=pentium4 -O2 -Demacs=1 -DHAVE_CONFIG_H 
>> -I../lib -I../nt/inc -DHAVE_NTGUI=1 -DUSE_CRT_DLL=1 -DPURESIZE=5000000 
>> -DPURESIZE=5000000 -o oo-spd/i386/make-docfile.o make-docfile.c
>
> I have no idea why that happened; lib-src/makefile definitely includes
> "-I../src" through the LOCAL_FLAGS variable.  Could it be that you
> somehow set LOCAL_FLAGS in the environment?
>
> In general, the compilation should never leave anything in the
> environment, so I don't think your explanation is quite right.
>
> Also, there's no need to "make bootstrap" twice; once you ran
> bootstrap once, the subsequent builds should be doe with just "make"
> in the nt/ subdirectory.
>
>> To get the build to work I had to use a fresh shell and explicitly set
>> SHELL=cmd.exe when invoking make
>
> Do you have a sh.exe somewhere on PATH?  Or do you have SHELL in your
> environment set to something other than cmd.exe?  Otherwise, there
> should be no need to set SHELL.
>
> Not that all this matters, now that you've succeeded to build.
>
> The only thing that does matter is that you've compiled with
> optimizations (which is the default), which will make it hard to
> examine the code under the debugger.  So I suggest to reconfigure and
> rebuild without optimizations, like this:
>
>  cd nt
>  configure --no-opt --enable-checking
>  make
>
> You will then have an unoptimized emacs.exe in src/oo/i386/emacs.exe.
> If you want it in bin/, say "make install" after the last "make".  The
> old optimized build will still be available in src/oo-spd/i386.
>
> Sorry, I should have mentioned this to begin with.
>
> And thanks for working on this.

I think I have a MSYS sh.exe somewhere in my path which caused the
build to fail if I didn't explicitly set SHELL.

Anyway, I managed to do some debugging and your theory seems to be
correct. The WM_KEYDOWN is passed to TranslateMessage which posts a
WM_CHAR with a question mark, so Emacs never sees the correct Unicode
character.

Also, the documentation for RegisterClass at
http://msdn.microsoft.com/en-us/library/windows/desktop/ms633586%28v=vs.85%29.aspx
says:

"If you register the window class by using RegisterClassA, the
application tells the system that the windows of the created class
expect messages with text or character parameters to use the ANSI
character set; if you register it by using RegisterClassW, the
application requests that the system pass text parameters of messages
as Unicode."

So I'll try to change to using RegisterClassW which should fix this
issue but might break other stuff since more messages will deliver
Unicode text then. An alternative might be to compare the result of
ToUnicode and ToAscii inside WM_KEYDOWN and skip calling
TranslateMessage when it will lose the correct character.

I'll get back to you with the results.