Re: highlighting non-ASCII characters

[Ted Zlatanov]

On Tue, 30 Mar 2010 07:04:05 -0700 "Drew Adams" <address@hidden> wrote: 

DA> Perhaps you're assuming that the code will be used in Emacs, so you say that
DA> Emacs treats all such quotes similarly or highlights them anyway etc. (so no
DA> problem).

DA> Emacs might be used to write raw documentation (e.g. including code samples)
DA> that is used to generate HTML or PDF or... Readers of that doc might then 
copy
DA> and paste such examples into an app other than Emacs for execution - an app 
that
DA> does not treat all such quotes similarly. 

DA> Just one hypothetical example, extrapolated from why we use straight quotes 
in
DA> our use of Framemaker.

I see what you mean.  It would require copy from a text mode (any
programming modes would show the problem with mismatched quotes).  I
don't know if Emacs should try to prevent this problem; the effort spent
is far larger than the potential benefit.

DA> Beyond that, I would think that there might be a number of use cases where 
one
DA> might want to visually distinguish characters that are difficult to 
distinguish
DA> - either exact homoglyphs or approximate ones. That's all.

So far we've discussed exact homoglyphs, but I think once we have the
basic UCS confusables we can augment it with more characters we think
are confusable.

DA> And there's the opposite potential problem: not distinguishing similar chars
DA> visually in the case where they do have different behaviors in some app. 
Suppose
DA> you prepare code (for example) in Emacs for use in some other context, and 
you
DA> want to be made aware when you use the wrong char, to avoid a problem
DA> downstream.

OK.

DA> Anyway, you get the point, I think. If you don't think there is a
DA> problem, I'm OK with that.

I think it's sufficient to start with the idnchars.txt and
confusables.txt databases (the latter parsed into an auxiliary hashtable
as well, to find what homoglyph group a character belongs to).  From
what you have said, it seems useful to augment the confusables.txt
database with "similar but not exact homoglyphs" eventually but it's not
as important.

I'll try to put something together soon, unless Lennart beats me to it,
to at least read the databases and generate the category-tables.

Ted