Re: Friendly discussion about (package-initialize)

[Radon Rosborough]

> > > > The problem is that even if I don't use package.el, there may
> > > > be some stuff left in ~/.emacs.d/elpa from previous times.
> > >
> > > Then don't do that:
> >
> > I'm actually not sure what "that" is referring to here.
>
> "That" refers to leaving stuff in ~/.emacs.d/elpa when you're not
> using package.el.

As an end user, why should I have to know how package.el stores its
data in ~/.emacs.d? Isn't that an implementation detail? I'm concerned
that knowledge of this information is apparently required to prevent
package.el from taking actions all on its own (without being asked)
after you've switched to another package manager.

At a bare minimum, there needs to be a prominent notice about this in
the documentation [1]. But I still maintain that the entire situation
is very sketchy.

> Maybe we should provide a template init file.

I think this is a good idea not just to solve this problem with
package.el, but also to make it easier to nudge new users towards
Emacs configuration best practices without having to change defaults
(which, as we all know, is extremely contentious).

> [ This file should be basically empty (tho could contain lots of
> comments. ]

That can be debated later. At the minimum, though, it would have to
contain a (package-initialize), with lots of accompanying comments to
explain what it's doing there and where you should put your code to
configure packages.

> But auto-creating this file just because you start Emacs is also a
> problem (similar to the auto-editing of this file you're complaining
> about).

Agreed. However, it can't possibly be worse that the current
situation, right? Currently, a new file is created automatically,
*and* any existing one is modified. My proposal would limit this
behavior to automatic creation, and not modification.

Of course, the init-file shouldn't be created at startup in 'emacs -Q'
mode.

> We also want to cater to old users of Emacs (who already have their
> own ~/.emacs) who start to use package.el.

I do like it when things "just work", but there's only so much magic
you can do to that end before you start losing things like robustness,
predictability, and your sanity.

I know I've brought this up before, but can you name a single other
program that deals with the problem of bad default behavior by
automatically modifying a free-form user-written configuration file on
startup, depending on a number of undocumented heuristics? If nobody
else does it, I'm sure there's a reason for that.

I like to think that most everyone who uses Emacs is intelligent
enough to understand a simple concept like "the package manager must
be initialized before you can use the packages". The problem is that
this simple concept is made quite complicated by all the magic of
package.el, and it is not clearly documented at all. If we had better
communication with users, then there would be no need for magic.

And before you say that some people don't want to modify their
init-file, and would prefer to rely on interactive commands only, let
me point out that this problem only arises if you try to put package
configuration in your init-file.

> Which functions do you use accidentally?

I didn't mean that I use package.el functions accidentally. I mean
that I use package.el functions intentionally (when I am testing other
people's package managers, for example), but I accidentally forget to
define advices on internal package.el functions before doing so, with
the result that my init-file is modified for no reason and I have to
go and revert it.

> I wonder what you mean by "break".

I mean that it inserts a call to `package-initialize' in my init-file
when that call is both superfluous and actively harmful due to the
particulars of my setup. It objectively does the wrong thing in my
case; that is why I said "break". Not to imply that this isn't the
right thing to do in general, but it's definitely not always the right
thing to do.

> > > > > We could change tactic: only call package--ensure-init-file
> > > > > when the user installs a package.
> > > >
> > > > I don't think I would enjoy this either.
> > >
> > > Do you mean that it would be worse, or that it would be better but
> > > still not good enough?
> >
> > Better but still not good enough.
>
> Great. Then let's try to do that. I suggest you M-x report-emacs-bug
> and request this change

Will do, although I'll probably wait to see where this discussion goes
first, so I have the proper context.

> I don't see why it would have such an effect.

Because if we removed `package--ensure-init-file' subsequently, then
there would no longer be any automatic activation of packages during
init, except for new users who had a template init-file generated.

But I'm glad to hear that my concern was unfounded, and the removal of
init-file modification via `package--ensure-init-file' will not face
any more resistance than it does currently.

===== Something to consider =====

Let's say that one day, another package manager is accepted into Emacs
core. Users would have a choice between package.el and alternative.el,
whatever the latter may be. Does it still make sense for Emacs to
inject a whole bunch of magic into the init sequence, including
modifying the user's init-file without their permission under several
different conditions, catering specifically to package.el? Should
Emacs do this for both package managers? That doesn't make a lot of
sense, since of course the packages installed by the two might
conflict, and the user only needs one package manager. It's up to the
user to explicitly tell Emacs which package manager to use, by placing
(package-initialize) or (alternative-initialize) in their init-file.
That's the only reasonable solution; anything else is piling hacks on
top of hacks. Does this sound familiar?

===== An allegory =====

Let us travel to an alternate universe in which the snippet package
YASnippet was designed according to the same principles as package.el,
and it was included into Emacs core. What would this mean?

Well, firstly, there would be a function called `yas-initialize', and
you'd have to call it in order to load all the currently defined
snippets, and rebind <TAB> so that it would expand snippets if
appropriate.

Whenever you started Emacs, `yas-initialize' would be run
automatically, which would rebind <TAB> without asking you, and then
you'd get a comment inserted into your init-file:

    ;; Added by Yasnippet.  This must come before configurations of
    ;; installed snippets.  Don't delete this line.  If you don't want it,
    ;; just comment it out by adding a semicolon to the start of the line.
    ;; You may delete these explanatory comments.
    (yas-initialize)

You could prevent this by setting the variable `yas-enable-at-startup'
to nil, but this wouldn't help you in the case that your init-file
encountered an error somewhere, or any number of other interesting
edge cases. It would only take one time for that comment to get
inserted, and then Yasnippet would start getting activated at startup
again, even if you had previously tried to disable it.

Of course, being a highly capable Emacs user, you are very careful to
never encounter any of those situations... somehow. But one day, you
decide to try out Yasnippet, to see if you like it. You set up the
configuration just how you like it, in a sub-module of your
modularized init-file. But as soon as you load Yasnippet, that comment
is back again, and in the wrong place! After an hour or two of poring
through the source code, it looks like the only way to permanently
disable this behavior is to place an advice on an internal function.
You can hardly believe your eyes.

"Surely the Emacs developers didn't think that everyone needed
Yasnippet activated in just the same way, or even that everyone wants
to use Yasnippet, right? This must be a mistake. How could anybody
think that globally rebinding <TAB> on startup is a good idea?"

So you go and complain about the situation, only to be told things
like:

> Well, the rebinding of <TAB> won't make any difference if the user
> doesn't use Yasnippet (i.e., doesn't have any snippets defined).

> We want to make it as straightforward as possible for users to
> define and use snippets.

> People who object to this kind of thing are in the minority; it's
> more important to make the experience as seamless as possible for
> Yasnippet users.

> Just make sure to delete all your snippets whenever you're done
> using Yasnippet, that will solve the problem.

I'm being intentionally facetious here; I certainly don't intend to
give offense. I just think that looking at the situation from outside
the lens of package.el will lend some good perspective. After all, no
package (and even no package *manager*) other than package.el does
this kind of thing, and I've yet to hear a reason why package.el just
*has* to be different.

Best regards,
Radon

P.S. If you think that comparing package initialization to rebinding
<TAB> globally is unfair, I'd say on the contrary it's quite an
egalitarian comparison:

slow init time due to loading unneeded snippets
  <==>
slow init time due to loading duplicate packages

conflicts between <TAB> rebinding and user's bindings
  <==>
conflicts between stale package.el packages and user's packages

unpredictable behavior of <TAB> because installed snippets are not
synced between machines
  <==>
unpredictable behavior of Emacs config because installed packages are
not synced between machines

disrespect of the user's desire to use another snippet manager
  <==>
disrespect of the user's desire to use another package manager

P.P.S. I originally started this comparison as a joke, but I'd
honestly be less offended by the <TAB> thing.

[1]: 
https://www.gnu.org/software/emacs/manual/html_node/emacs/Package-Files.html