Re: Python 3 and pygments-parser

[Marcus Harnisch]

Hi Shigio

Never mind, the idea with ‘locale.getpreferredencoding(False)’ isn't going to work anyway, for the lack of multibyte support and differences between user locale and e.g. filename encoding, etc.

What about converting the wrapper to consistently use bytearrays instead of strings? The original problem that made me look into this was that ‘load_ctags_path()’ currently returns a bytearray (non-Win32) and the variable ‘UNIVERSAL_CTAGS’ is later compared with a string ('' != b''). Under Python 2 this didn't matter.

Best regards,

Marcus

On Wed, May 8, 2024 at 2:17 AM Shigio YAMAGUCHI <shigio@gnu.org> wrote:

Hi Marcus,

> The method ‘load_ctags_path()’ doesn't work with Python 3
> (unless perhaps with Win32/latin1).
> In other places the encoding has been fixed to ‘latin1’ as well.

It doesn't matter that the encoding is latin1, because Global
doesn't support multi-byte code set.

        [FAQ]
        Q10. Does Global support multi-byte code set?
             Which character code set is supported?

        A10. Global doesn't support multi-byte character code set yet.
             Global supports only ASCII and ASCII super-sets.

> At least for ‘load_gtags_path()’ I'd probably change the encoding
> to the current locale with ‘locale.getpreferredencoding(False)’.

What does it accomplish?

> Python 2 support has been dropped from Pygments more than
> four years ago. Does it make sense to maintain Python 2
> compatibility in the pygments-parser?

Though maintaining compatibility is important, if we can achieve
something that users will be happy with, it is not the top priority.
I'd appreciate it if you could let me know what you're trying
to achieve.

Regards,
Shigio
--
Shigio YAMAGUCHI <shigio@gnu.org>
PGP fingerprint:
26F6 31B4 3D62 4A92 7E6F  1C33 969C 3BE3 89DD A6EB