Re: user-controlled load-path extension: load-dir

[Stephen J. Turnbull]

Ted Zlatanov writes:
 > On Thu, 10 Mar 2011 02:29:02 +0900 "Stephen J. Turnbull" <address@hidden> 
 > wrote: 
 > 
 > SJT> I can only imagine that a load-dir would very likely be the source
 > SJT> of numerous bugs as some snippets conflict with or depend on others
 > SJT> but the automatic loader gets them in the wrong order.
 > 
 > We're just adding a mechanism for easy personal code deployment and
 > modularization, not a cure for broken code.

There's no broken code here, just code not engineered with automatic
loading in mind.  Ie, it's the idea of automatically loading random
snippets that is broken: it's a mechanism for speeding up the mess, as
Robert Townsend put it.  I don't think that belongs in core, but
rather should be an individually maintained function, tuned to each
person's requirements.

As for easy personal code deployment and modularization, there's
nothing that says that deployment and modularization requires
implementation as separate files, and in fact using separate files for
most things that you would do in an init file is an annoyance as
Stefan points out.  Stefan's recommendation of using outline minor
mode easily scales to a 100kB file for me in other contexts, I imagine
it would work fine for the init file.  While I admit YMMV as a user,
from the point of view of Emacs maintainers this blows away the
load-dir idea in terms of maintainability of Emacs core.

Note that the good reasons for using separate files (assuming you use
an editor that supports things like outline minor mode -- you do,
don't you?<wink>) are not really compatible with automatic
installation and loading.  They're things like lazy loading and
optional loading to avoid memory bloat and namespace pollution.  Lazy
loading could be a win (somebody mentioned autoloading instead of
automatic loading at init time), therefore, but guess what? that
requires reading code and deciding where to put autoload cookies.

 > SJT> Note also that .d directories generally use some convention (such as
 > SJT> file names starting with fixed width integers) that ensure that
 > SJT> snippets sort in the right order.
 > 
 > I'm sure users can use that (I will).  But they can also specify the
 > order with explicit loads if that's what they need.

Neither of those is easy or automatic.  To the extent that such are
needed, people are going to have to sit down and read code to figure
out what the problems are.  That's precisely what the system is
supposed to avoid.

Note that you also have a problem that such explicitly loaded files
will often need to be moved out of the load-dir, as they won't be
designed to be idempotent.  Mostly not a problem, of course, and
therefore all the more annoying and confusing when it does arise.
 
 > SJT> More modern systems use (please sit down, you're in for a shock)
 > SJT> full-blown dependency systems (requires, provides, conflicts, etc)
 > SJT> in the snippets, which avoids the need to maintain explicit order
 > SJT> in favor of a partial order.
 > 
 > That's, again, overengineering the problem (as Mark Twain put it, "with
 > all the modern inconveniences").

Well, no, it's not.  The problem in question is maintaining a .d
directory without requiring the user to have a clue.  If Stefan and
Yidong could get away with "Ted Z says that's overengineering so we
ain't gonna touch it", it might be worth putting this in core.  But my
recommendation to them is "snippet conflicts are going to be a
persistent annoyance in the long term; let Ted Z and Jan D maintain a
package and deal with the PEBKACs and RFEs."