bug#22038: 25.1.50; Character folding issues with isearch

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Cc: 22038@debbugs.gnu.org
> Date: Sat, 28 Nov 2015 18:10:53 +0100
> 
> > (That's the only way I could parse "multiple characters matching a
> > single string".)  We will have that, but it won't allow "ss" to match
> > "ß", unless you customize character-fold-table to include that.  The
> > reason is that "ß" doesn't have any decompositions in the Unicode
> > database, so the default character-fold-table doesn't include any
> > expansions for it.
> 
> This suggests to me that basing character folding solely on character
> decomposition is insufficient.  From a user's point of view I see no
> reason why the search string "a" under character-folding matches "ä" but
> not e.g. "æ".  Requiring a customization to get the latter strikes me as
> a user-unfriendly crutch to work around a deficient implementation.  (I
> don't know if it's easy to improve, I'm just giving my impression as a
> user.)

Easiness is not the most important issue here: there's a more basic
problem involved.  Both "ß" vs "ss" and "æ" vs "a" (or "ae") are
language-specific: they are only valid matches in the context of
specific languages.  AFAIU, that is why they are not in the Unicode
database.  And we don't yet have language-specific text processing
capabilities and infrastructure (well, string-collate-lessp and
string-collate-equalp are a beginning, but only that).  So allowing
those by default risk running afoul of what users want.

There are more language-specific foldings possible, outside of the
European languages.  For example, folding of Arabic positional forms
of the same letter.  These are at times much more important than the
above ligatures, and yet we don't support them yet, either.

In this initial release of such functionality I think it is prudent to
go by the standard, because we don't yet have any real-life experience
to build upon.  That doesn't cover every possible use case where a
more radical folding would be useful, but we had nothing in Emacs 24,
so this is still a large step in the right direction, IMO.  Let's not
bite more than we can chew.