bug#20316: 24.5; `string-lessp' doesn't respect value of LC_COLLATE

[Eli Zaretskii]

> From: Alexis <flexibeast@gmail.com>
> Date: Mon, 13 Apr 2015 16:42:30 +1000
> Cc: michael.albinus@gmx.de
> 

>         (getenv "LC_COLLATE") => "hr_HR.UTF-8"
>         (string-lessp "Ć" "D") => nil
>         (string-lessp "Đ" "S") => nil
>         (string-lessp "Š" "Z") => nil 
> 
>     `string-lessp' is not design to respect collation order. The docstring 
> speaks about lexicographoc order. 
> 
> i'm sorry, i don't follow this. Yes, the docstring talks about lexicographic 
> order. But i think it's reasonable for users to expect Emacs' built-in sort 
> functionality (e.g. `sort-lines') to respect their current locale such that 
> lexicographic order and collation order are treated as basically synonymous. 
> Unless there's an important distinction between the two in this context that 
> i'm missing?

Emacs is a multi-lingual editor, and it isn't clear how to apply
locale-specific settings, including collation, to text that can
potentially include many scripts.  E.g., a locale could specify a
codeset that doesn't cover characters outside of a particular script,
which will produce undefined results if you try using system sorting
routines with characters outside of that single script.

Also, using locale-specific sorting would produce different results
not only in different locales, but also on different platforms in the
same locale, because the implementation of locale collation order
differs from platform to platform and from one C library to another.

Please also note that locale-sensitive collation order could mean more
than just order of characters.  E.g., it could specify that
punctuation characters or differences in accents should be ignored.

So by default, Emacs sorts disregarding locale-specific ordering,
basically using the Unicode codepoints of the characters to order
them.

> In this specific case, a user of a package i maintain was surprised when 
> `org-sort' didn't sort their Croatian-language Org entries properly. Under 
> the hood, `org-sort' uses `string-lessp'. Is the bottom line that they'll 
> have to wait until at least 25.1 before Org could add conditional code to use 
> `string-collate-lessp', to at least DTRT for users of >25.0?

You could use the external 'sort' utility in the meantime, if it
supports locale-dependent ordering.  Once again: the results will be
not 100% deterministic, even for the same locale, so your users should
"caveat emptor".

May I ask what does that package of yours do that it needs
locale-dependent sorting?