bug#30078: 27.0.50; Use lexical-binding for M-:

[Stefan Monnier]

> I figured M-x ielm would qualify as well.

Seeing how forcing lexical-binding within *ielm* raised some objections,
I've changed my patch so it only sets lexical-binding in the *ielm*
buffer but the evaluation mode will still depend on the
lexical-binding setting in ielm-working-buffer (so evaluating
`lexical-binding' will again return a value which indicates whether
expressions are evaluated using lexical-binding).

New patch below.

There were some issues regarding the general long term goal of
standardizing on lexical-binding.  So I'm restarting this discussion
below.

I wrote:
> > I very much hope to see Emacs shed its dynbind-Elisp support at some
> > point, tho it still seems to be a pretty distant future and I'm not even
> > sure how we'll get to that point.

To which Eli replied:
> It's quite clear that these are your views, but going in that
> direction should be an explicit project-wide decision, widely agreed
> and accepted by main project contributors.  Doing it one subtle step
> at a time is not the right way, IMO.  If and when such a decision is
> made, we would not need to have these discussions, we could simply go
> lexical-bind as quickly as it's practical.

Fully agreed.  FWIW, this is the decision I took when I added
lexical-binding into Emacs-24.1.  I had no idea that someone might like
to disagree: the lexbind-Elisp language is a strict superset of the
dynbind-Elisp language and for that reason I don't see any reason other
than backward compatibility to keep dynbind-Elisp.

I'm not trying to fool people by "doing it one subtle step at a time",
I'm only trying to make this switch as painless as possible.  I don't
yet have a plan for how we'll get rid of dynbind-Elisp and I'm sure
it'll take many years and several more steps, some of them less subtle
than others.  Some of the steps I could imagine introducing in the not
too distant future:
- finish converting all Emacs's .el files to use lexical-binding:t.
- emit a warning when opening an Elisp file which doesn't specify
  `lexical-binding:t' in its local vars.
- emit a warning when `eval` is called without a nil 2nd arg.

AFAICT by looking at packages on github, lexical-binding is very
popular, despite the fact that it limits packages to Emacs≥24 and
requires explicitly adding a magic cookie on the first line.  Also I've
only gotten positive feedback so far about it (more specifically, the
above paragraph of yours is the most negative feedback I've hard about
lexical-binding so far).

Eli also wrote:
> Do we know for a fact that most users who extend Emacs by writing
> their own Lisp have background in Lisps and other similar languages
> where lexical binding is the default?  If not, dynamic binding might
> be easier to grasp for the newcomers.

Lexical scoping is not specific to Lisp or similar languages: it's the
scoping supported by virtually all programming languages such as
Javascript, Pascal, Fortran, Smalltalk, C, Python, Rust, Java, C#,
Clojure, Ruby, Perl, ...  The vast majority of those *only* supports
lexical scoping.

It's dynamic binding which is the odd one out: I can't think of too many
languages supporting dynamic scoping other than Scheme, Common-Lisp,
Elisp, Logo, sh, and Perl (and maybe Awk tho the doc I can find doesn't make
it clear), most of which also support lexical scoping (the only
exceptions I know are Logo, sh, and maybe Awk).


        Stefan