Re: per-buffer language environments

[Stephen J. Turnbull]

Eli Zaretskii writes:
 > > From: "Stephen J. Turnbull" <address@hidden>
 > > Cc: Kenichi Handa <address@hidden>,
 > >     address@hidden
 > > Date: Tue, 14 Dec 2010 20:38:43 +0900
 > > 
 > > Eli Zaretskii writes:
 > > 
 > >  > > * Which coding systems have higher priority when inserting a
 > >  > >   file in the current buffer.
 > >  > 
 > >  > I could understand how the font selection and the default input method
 > >  > are related to the language, but what do encodings have to do with
 > >  > that?  The preferred encoding is generally an attribute of a locale,
 > >  > not of a language.
 > > 
 > > Note the word "insert", which implies "read".  It is certainly true
 > > that a locale may specify an encoding.  However, if the person is
 > > Japanese, they may specify ja_JP.UTF-8 for their locale and strongly
 > > prefer that files be written with that encoding, yet still need to
 > > read files in other encodings. [more examples snipped]
 > 
 > Those are all valid concerns, but they are just the tip of an
 > iceberg.

No, they *are* the iceberg, at least as far as the autopilot is
concerned.  After that, you *must* ask the user.

 > There's an almost infinite number of combinations of a language and
 > the preferred encoding

Sure, but given a language and the set of encoding features Emacs
knows how to detect *when reading from a stream*, there remains
substantial ambiguity.  Setting the priority list can remove almost
all of that ambiguity, leaving what's left for the user.  That is what
the priority lists are for, and it is a useful feature of the language
environment.  All problems with the language environment that I know
of stem from its global nature applying to all buffers and the
application itself, not from appropriate use in a given buffer.  IOW,
it's just the defects of the POSIX_ME_HARDER locale mirrored into
Emacs itself.

The preferred encoding, OTOH, is a heuristic for the encoding of files
read, and the default for the encoding of files written.

These two are independent in principle, but of course "preferred
encoding for writing" = "highest priority encoding for reading" is a
very valuable heuristic.

 > , and it's impossible to fold them all, or even their significant
 > fraction,

Of course a significant fraction is possible.  That's precisely what
the priority lists have been achieving since the early 1990s.  If your
complaint is that we should do better, "patches welcome" is the only
thing I can think of to say.  But it does a pretty damn good job
already, and buffer-local language environments should cut current
damage by 80% or more; your work is cut out for you.

 > in a reasonably usable user-level interface.  We shouldn't even
 > try, IMO; we already have prefer-coding-system

Huh?  prefer-coding-system has two effects: it promotes a certain
coding-system to highest priority in its category, and it promotes
that category to highest priority in case of ambiguity.  IOW, it's a
user override of the priority setting that comes from the language
environment.  A completely different purpose (handling exceptions)
from the language environment itself (handling the unmarked case).

Are you sure you have any idea what you're talking about?  (That's an
honest question; the way you are going, I have to wonder.  If you say
"yes", I'll trust you, but I'd appreciate an explanation of what
you're talking about that refers to real bugs in the current system,
rather than general features that offend your sense of design.)

 > , the coding: cookies, the .dir_locals meta-data,

Speaking of *my* sense of design, two features that are an offense
against Man and a stench in the nostrils of God.  But I digress.

 > etc. to cover the situations where the user knows what encoding should
 > be preferred/used, even though her language and locale say otherwise.
 > 
 > set-language-environment accepts a single string, which should be a
 > language name, as its argument.  (There are some "languages" that we
 > recognize, such as "Chinese-GB18030", which sneak in the encoding as
 > well, but that's an anomaly, I think, which goes back to when Emacs
 > didn't have locale environments to express that.  Now that we do, we
 > could get rid of that, at least in principle.)  Therefore, a language
 > environment should set the defaults suitable for the language, and
 > that doesn't include the encoding, or at least does not have to fit
 > each minor cultural variant of the language.

That's not what coding priority settings are for.  They are to remove
ambiguities like "we have EUC, but which one?" and "we have
Windows-125x, but which one?" and "since ISO-8859-1 allows all 256
bytes, if we want to give priority to Chinese or Japanese, that had
better come late in the list!"

 > >  > The fact that we mix them is because Emacs had
 > >  > language environments before it had locale environments.
 > > 
 > > What's a "locale environment"?
 > 
 > See set-locale-environment.

"[No match]"

YAGNI, apparently.  (For values of "you" == "me", obviously.  YMMV. :-)

 > > AFAIK Emacsen use the locale as a heuristic for determining the
 > > language environment
 > 
 > There's no heuristic involved, AFAIR.  Emacs has a database of
 > languages _and_encodings_ suitable for the known locale names.

You're confusing "algorithmic" with "non-heuristic".  Of course it's
possible to have a heuristic algorithm.

And of course in this case, locale is a heuristic.  *Emacs is a
multilingual* (well, technically, multiscript) *application*, and any
setting of the language environment that doesn't take into account the
current text we're working with is surely heuristic.

 > set-locale-environment uses that database to get the language and the
 > preferred encoding(s), then calls set-language-environment with the
 > language, and sets the priorities of the encodings according to the
 > encoding preferences.

That's an unnecessary API, ISTM.  (set-language-environment nil)
should do that.  Perhaps there should be a `set-locale' command to
override the POSIX_ME_HARDER locale taken from the environment, but
the POSIX_ME_HARDER locale is an abomination in a multilingual
application and should be buried as deeply as we can manage.  It is,
of course, a useful heuristic for the user's preferred language
environment for *scratch*, but that's about as far as we can take that.