Re: Proposals for fixing package-initialize

[Clément Pit-Claudel]

Incredible work. Thanks a lot!

On 2017-09-04 07:31, Radon Rosborough wrote:
> Hello all,
> 
> You may have seen slightly over 150 emails fly by in the last few
> weeks talking about package-initialize. Hilariously, we still haven't
> managed to reach a conclusion.
> 
> This email is an attempt to present all the proposals that have been
> discussed, in an even-handed way, so that we can make a decision
> without everyone needing to have read the aforementioned 150 emails.
> 
> I will *not* be providing any context on the issue; see [1] for that,
> but note that the discussion branched out so much that everyone's
> positions are now completely different :P
> 
> ==> Proposal A: Leave it alone
> 
> Summary:
>   - call package-initialize in startup.el after loading the init-file
>   - when package-initialize is called, also insert a call to it into
>     the init-file, if one was not found there
>   - this is the current behavior
> 
> Advantages:
>   - requires no work
>   - people are used to the current behavior
>   - backwards compatible
> 
> Disadvantages:
>   - modifying the user's init-file is ugly
>   - doing it automatically is annoying and possibly dangerous
>   - if the user has customized package.el, then Emacs will put
>     package-initialize in the wrong place
>   - this type of behavior is not common practice
> 
> ==> Proposal B: Add a second init-file
> 
> Summary:
>   - add a second init-file, ~/.emacs.d/early-init.el
>   - this file is sourced earlier than init.el, during startup.el
>   - move the call to package-initialize in startup.el from its current
>     location to between early-init.el and startup.el
>   - don't have package.el modify the init-file ever
> 
> Advantages:
>   - does not require modifying the user's init-file
>   - package.el still works out of the box
>   - the only people affected by the change are the ones who have
>     customized package.el
>   - foolproof, never does the wrong thing (except for advanced
>     existing configs that customize package.el)
>   - people can also use early-init.el to disable things like the tool
>     bar before they are loaded in the first place: two birds with one
>     stone
>   - the only people who need to know about early-init.el are advanced
>     users who want to customize package.el
>   - Customize works both for customizing package.el and also for
>     customizing packages
>   - does not require package-initialize to be put in the user's
>     init-file, not even manually
>   - makes it easy to disable package.el permanently
> 
> Disadvantages:
>   - adds another init-file, which is additional complexity
>   - we must add a mechanism to Customize that allows to specify the
>     file into which a particular variable should be saved, so
>     package.el can still be customized
> 
> ==> Proposal B': Add an early-init special form
> 
> Summary:
>   - same as above, but instead of a separate file, allow init.el to
>     start with a special (with-early-config ...) form
>   - if with-early-config form is found in init-file, load it when
>     early-init.el would be loaded
> 
> Advantages:
>   - we still only have one init-file
> 
> Disadvantages:
>   - the startup sequence is a little weird
>   - it's not clear how this would work with Customize
> 
> ==> Proposal C: Add a template init-file
> 
> Summary:
>   - if Emacs is started without an init-file, generate one from a
>     template
>   - the template init-file has a call to package-initialize in it, and
>     some comments explaining where to put package and package.el
>     configuration
>   - the template init-file is not generated in 'emacs -Q' mode
>   - get rid of package.el modifying the init-file
> 
> Advantages:
>   - does not require modifying the user's init-file
>   - package.el still works out of the box for new users
>   - we could use the template init-file for encouraging users towards
>     better defaults without breaking backwards compatibility
> 
> Disadvantages:
>   - having Emacs automatically create files is not ideal
>   - if we decide against using the template init-file for better
>     defaults, then it's kind of awkward to just use it for package.el
>   - doesn't work for users with existing init-files
>   - still reqiures a call to package-initialize in the user's
>     init-file
> 
> ==> Proposal D: Hook into error handlers
> 
> Summary:
>   - call package-initialize whenever a void-function or require error
>     is encountered
>   - get rid of package.el modifying the init-file
>   - possibly get rid of call to package-initialize in startup.el?
> 
> Advantages:
>   - doesn't require modifying the user's init-file
>   - probably package.el still works out of the box
> 
> Disadvantages:
>   - implications (performance, correctness, implementation details)
>     are completely unexplored
>   - not very elegant
>   - business logic from the package manager is intermixed with a
>     fundamental, generic part of the Elisp interpreter
>   - hard to tell when package-initialize gets called, so best practice
>     would probably still be to have package-initialize in the
>     init-file
>   - the fact that a core Lisp error causes lots of magic to happen
>     will make debugging more "fun"
> 
> ==> Proposal E: Call package-initialize multiple times
> 
> Summary:
>   - add a second call to package-initialize before init.el is loaded
>   - if package-initialize is called again, and package-load-list or
>     other variables have been customized since the last time it was
>     called, unload any packages that were previously loaded that now
>     should not be loaded
> 
> Advantages:
>   - does not require modifying the user's init-file
>   - backwards compatible, at least if it works
> 
> Disadvantages:
>   - impossible to disable package.el without introducing a magic
>     dotfile (~/.emacs.d/.nopackage) or similar
>   - impossible to unload packages correctly, in general, so this
>     approach would be buggy in many edge cases
>   - severe performance regressions if many packages are disabled using
>     package-load-list
>   - "As I said, there's no doubt that do+undo is a poor way to do
>     nothing." "Again, here we're back to the obvious observation that
>     do+undo is a poor and complicated way to do nothing." "Can't
>     disagree that "do+undo" is a poor way to do nothing." -- Stefan
>   - requires extra work to make multiple calls to package-initialize
>     faster, not to mention figuring out how unloading is supposed to
>     work, or if that's even possible in principle
> 
> ==> Proposal F: Use more declarative configuration
> 
> Summary:
>   - make it so that packages can be configured in a way that causes
>     package-initialize to be called automatically
> 
> Advantages:
>   - would solve all the problems
> 
> Disadvantages:
>   - nobody has any idea how to do this
>   - and it would probably take several years anyway
> 
> ==> Proposal G: Parse the init-file
> 
> Summary:
>   - call package-initialize in startup.el before loading init.el
>   - parse the init-file in order to find out the values of
>     package-load-list, etc. before running package-initialize
> 
> Advantages:
>   - in the cases that it works, it would allow package.el to work out
>     of the box without the need for package-initialize in the user's
>     init-file
>   - Emacs would not need to modify the user's init-file
> 
> Disadvantages:
>   - would require solving the halting problem to work in general
>   - is a horrifying hack
> 
> ==> Proposal H: Make the user do it manually
> 
> Summary:
>   - make it so that package.el doesn't modify the init-file, but don't
>     change anything else
>   - this is how it worked before the first "solution" to this problem
>     was implemented
> 
> Advantages:
>   - Emacs does not modify the user's init-file
> 
> Disadvantages:
>   - package.el does not work out of the box, since package
>     customizations will fail when put into the init-file
>   - consequently we get lots of superfluous bug reports
> 
> ==> My position
> 
> I think that the above advantages and disadvantages speak for
> themselves, but to be clear, I think Proposal B is the best. It has
> basically no practical disadvantages, solves every problem that is
> tangled up in this issue with zero migration necessary except for very
> advanced users, is trivial to implement, and simultaneously solves a
> large class of other problems related to disabling things that are
> enabled early during startup. Every other proposal has significant
> practical disadvantages.
> 
> ==> Postscript
> 
> I'd like to say thank you to Eli for pushing the question of whether
> it would be practical to call package-initialize only in startup.el,
> Timur for questioning why a second init-file would be so bad, and
> Clément for explaining to me what was wrong with my original proposal.
> Their arguments resulted in the discussion of what I now think is a
> better proposal than my original one.
> 
> And yes, I really did review all 150 of those emails to gather
> information for this summary :o
> 
> Best regards,
> Radon Rosborough
> 
> [1]: https://lists.gnu.org/archive/html/emacs-devel/2017-08/msg00154.html
> 
>