supporting in the UTF-8 environment on native Windows

[Bruno Haible]

Lasse Collin reported in
<https://lists.gnu.org/archive/html/bug-gettext/2024-12/msg00111.html>
that the setlocale() override from GNU libintl does not support the
UTF-8 environment of native Windows correctly. That setlocale() override
is based on the setlocale() override from gnulib. So let me add that
support here.

What I call the "UTF-8 environment of native Windows" is a way of
packaging an application (details are in [1]) in such a way that
GetACP() return 65001, the codepage number for UTF-8.

In fact, there are apparently two variants of this mode:
  - the legacy Windows settings variant: when you haven't ever
    (or recently?) changed the system default locale of Windows 10,
  - the modern Windows settings variant: when you have changed
    the system default locale of Windows 10.
With the legacy Windows settings, the setlocale() function produces
locale names such as "English_United States.65001" or
"English_United States.utf8". With the modern Windows settings, it
produces "en_US.UTF-8" instead. (This is with both mingw and MSVC,
according to my testing.)

The various locale-related modules of gnulib were never tested in
the UTF-8 environment. This series of patches adds support for it,
with unit tests.

[1] 
<https://learn.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page>


2024-12-23  Bruno Haible  <bruno@clisp.org>

        mbrtowc tests: Test in the UTF-8 environment on native Windows.
        * tests/test-mbrtowc-w32utf8.sh: New file.
        * tests/test-mbrtowc-w32utf8.c: New file.
        * modules/mbrtowc-tests (Files): Add these files and
        m4/windows-rc.m4, tests/windows-utf8.rc, tests/windows-utf8.manifest.
        (Depends-on): Add test-xfail.
        (configure.ac): Invoke gl_WINDOWS_RC.
        (Makefile.am): Arrange to compile test-mbrtowc-w32utf8 and run
        test-mbrtowc-w32utf8.sh.

2024-12-23  Bruno Haible  <bruno@clisp.org>

        setlocale tests: Test in the UTF-8 environment on native Windows.
        * tests/test-setlocale-w32utf8.sh: New file.
        * tests/test-setlocale-w32utf8.c: New file.
        * modules/setlocale-tests (Files): Add these files and
        m4/windows-rc.m4, tests/windows-utf8.rc, tests/windows-utf8.manifest.
        (Depends-on): Add test-xfail.
        (configure.ac): Invoke gl_WINDOWS_RC.
        (Makefile.am): Arrange to compile test-setlocale-w32utf8 and run
        test-setlocale-w32utf8.sh.

        setlocale: Support the UTF-8 environment on native Windows.
        * lib/setlocale.c: Include <windows.h>.
        (setlocale_unixlike): In the UTF-8 environment, append a suffix ".65001"
        to the locale names passed to the native setlocale().

2024-12-23  Bruno Haible  <bruno@clisp.org>

        localename tests: Test in the UTF-8 environment on native Windows.
        * tests/test-localename-w32utf8.sh: New file.
        * tests/test-localename-w32utf8.c: New file.
        * modules/localename-tests (Files): Add these files and
        m4/windows-rc.m4, tests/windows-utf8.rc, tests/windows-utf8.manifest.
        (Depends-on): Add test-xfail.
        (configure.ac): Invoke gl_WINDOWS_RC.
        (Makefile.am): Arrange to compile test-localename-w32utf8 and run
        test-localename-w32utf8.sh.

        localename-unsafe: Support the UTF-8 environment on native Windows.
        * lib/localename-unsafe.c (gl_locale_name_from_win32_LANGID): Append a
        suffix ".UTF-8" to the result if GetACP() is UTF-8.

2024-12-23  Bruno Haible  <bruno@clisp.org>

        localcharset tests: Test in the UTF-8 environment on native Windows.
        * m4/windows-rc.m4: New file.
        * tests/test-localcharset-w32utf8.sh: New file.
        * tests/test-localcharset-w32utf8.c: New file.
        * tests/windows-utf8.rc: New file.
        * tests/windows-utf8.manifest: New file.
        * modules/localcharset-tests (Files): Add these files.
        (Depends-on): Add test-xfail.
        (configure.ac): Invoke gl_WINDOWS_RC.
        (Makefile.am): Arrange to compile test-localcharset-w32utf8 and run
        test-localcharset-w32utf8.sh.

        localcharset: Support the UTF-8 environment on native Windows.
        * lib/localcharset.c (locale_charset): Recognize also the special case
        of a setlocale() result that ends in ".UTF-8".

0001-localcharset-Support-the-UTF-8-environment-on-native.patch
Description: Text Data

0002-localcharset-tests-Test-in-the-UTF-8-environment-on-.patch
Description: Text Data

0003-localename-unsafe-Support-the-UTF-8-environment-on-n.patch
Description: Text Data

0004-localename-tests-Test-in-the-UTF-8-environment-on-na.patch
Description: Text Data

0005-setlocale-Support-the-UTF-8-environment-on-native-Wi.patch
Description: Text Data

0006-setlocale-tests-Test-in-the-UTF-8-environment-on-nat.patch
Description: Text Data

0007-mbrtowc-tests-Test-in-the-UTF-8-environment-on-nativ.patch
Description: Text Data