Incorporating caching into defgroup/defcustom/defvar for Emacs 25

[Sean Allred]

Hi all,

Since this is a long email (sorry!), an executive summary: many Emacs 
applications use data that is persistent from session-to-session. The mechanism 
for doing so varies from application-to-application, but there are a few 
packages on the ELPAs that promise this, notably cache, pcache, and the new 
stash.el (my own creation for sx.el, both on github). I’ve discovered a lot of 
overlap in the way I’ve designed stash.el and the way the defgroup/defcustom 
system already works, so I’m curious to see how open you all are about 
introducing caching functionality baked into Emacs. defgroup and defcustom 
aren’t so difficult, but defvar (being a C function) introduces concerns that 
I’ve yet to truly research / understand.

One of the reasons that makes Emacs so awesome (or at least interesting) is the 
variety and number of full-blown application written in/for Emacs. Gnus and Org 
are prime examples, and *nothing* would be the same (for me) without Ido. All 
of these applications use on-disk data to keep information from session to 
session. This need is hardly limited to Emacs internally, too; Artur and I have 
been at work on sx.el (on github), another moderately complex Emacs application 
that uses (and outright needs) on-disk data. Most veritable applications use 
on-disk data, and I can only guess that they all do so with an in-house 
mechanic. The use-case is there.

There are a few caching libraries on the ELPAs at this time: cache, pcache, and 
stash (my own creation for sx.el). In my honest opinion,

- `cache’ is too simple. For one, it doesn’t even save its data! It merely 
provides a means for cached data to expire - a necessary utility for full 
support.
- `pcache’ is too complex (for my taste). It’s not immediately apparent 
where/how the data is stored. (Disclaimer: I have absolutely zero experience 
with eieio, its sole dependency.) As an added :(, it’s not available on the GNU 
ELPA.
- `stash’ is, I think, a good compromise between the two, though some filling 
out has to be done on the featureset. For example, it currently does not 
support cache timeouts. SX hasn’t needed it yet, but I’m certain other 
applications would. I’m, uhhh, I’m working on it :)

Something I’ve noticed in writing stash.el (and discussing it with Artur a bit) 
is that I’m duplicating functionality. To see this, let me explain (briefly) 
how stash.el works:

- &optional Programmer defines an ‘application’. This application implies a 
subdirectory within the over-arching `stash-directory’ where data is saved at 
some programmer-defined interval.
- Programmer defines a `stash’, optionally belonging to an application, with an 
associated filename. This filename is where this `stash’ is saved to disk as 
interpreted by `prin1-to-string'. (A `stash’ is just a variable with a few 
symbol-properties as metadata.)
- ...
  From this point on, the programmer just sets the variable’s value normally 
(with set/setq/etc.). On an idle-timer (set up by defining the application - 
there is a ’null’ application set to a minute’s idle time), the entire 
application is written to disk.

There are a few flaws in this approach (for example, not every variable has 
changed from the time it was last saved), but it’s working out pretty well. The 
most glaring flaw in this design is the duplication of the group-variable 
association already available (or conceptually so) in the 
defgroup/defcustom/defvar system. Given its generally applicability, I think 
caching in this (or similar) fashion would be a good addition to the system via 
keyword arguments to those three macros:

    (defgroup sx nil
      "Customization group for the `sx' package."
      :prefix "sx-"
      :tag "SX"
      :group ‘applications)
     ; => Would prepare to create a folder called “sx” in 
`customize-cache-directory’
    
    (defcustom sx-init-hook nil
      "Hook run when SX initializes.
    Run after `sx-init--internal-hook'."
      :group 'sx
      :type ‘hook
      :cache “init-hook.el”)
     ; => Would prepare to create a file called “init-hook.el” in 
`customize-cache-directory’/sx
    
    (defvar sx-variable :group ’sx :cache t :expire 60)
     ; => same as above, but basename defaults to “sx-variable” set to expire 
after one minute of inactivity

It’s this last that I’m most concerned about: `defvar’ is defined in the C 
source code. I haven’t taken the slightest peek at Emacs’ C sources, but I’d 
reckon there wouldn’t be a simple analog from the approach I took in stash.el. 
I’m just not sure. Perhaps the functionality can be moved to a Lisp defvar*.

I see a conceptual overlap with Customize’s own managed `custom-set-variables’ 
form, but they aren’t quite the same thing. It’s worth pointing out the 
similarity, but I’m not sure what to make of it.

I know this is a long email, so many thanks for reading it through. I’d really 
like to know interested thoughts on the topic. I’m perfectly willing and able 
to implement the functionality (or at least do most of the legwork) pending the 
necessary FSF paperwork, but it would be a non-trivial undertaking in an 
already busy schedule. Moreover, I’m sure there are multiple considerations 
I’ve missed :)

Best,
Sean Allred