RE: Changed outside --> set, in Customize UI

[Drew Adams]

Note: In this reply I also provide some of the reasons for the new
approach outlined in my separate mail, Subject: New tack for
Customize. See, for instance, the distinction below between different
types of library and how to treat them.

        One problem mentioned is that trying to save a
        "changed-outside" option might not "work": some library you
        load might change the saved value to something different after
        you restart. Well, _setting_ an option in Customize and then
        saving it might not work either - for exactly the same reason.
    
    I don't understand the argument there.  What is the secons
    scenario?

This argument can be dropped. It's really the same problem, and it's
only a minor point: a user can see "Set" or "Unchanged" (saved) for an
option at some point, and so perhaps feel reassured, and yet still
have the same problem when you restart Emacs.

You set an option in Customize, save it, then restart Emacs. Its value
in Customize is different from what you saved, because of an outside
change by a library. It makes no difference whether or not the value
you saved was first "Set". Set values and changed-outside values can
equally be overridden by subsequent outside changes. Just because you
set an option, so you don't see the "warning", is no guarantee that
the problem won't arise.
    
         When changing a defcustomed variable from a function, that
         function should be interactively called by the user
    
       Why?
    
    To avoid confusion.  In general, it is a good idea to separate the
    values that users set from the values that programs set.

Yes, but those values don't differ in kind. The question is "_How_ can
a user set a value?" If a user uses a program to set a value,
intentionally, is it a user-set value or a program-set value?

Pick a level of unity for the "Set" (vs "changed-outside") club:

1. Set inside Customize.
2. #1 + `set-variable' used interactively.
3. #2 + any command called interactively that has as purpose to change
        the value. 
4. #3 + any interactive command, used interactively or not.
5. #4 + any function or sexpr executed interactively (`M-:').
6. #5 + any function or sexpr called by any function or sexpr in #5.
7. #6 + explicitly loading any library (interactively).
8. #7 + loading any library in your custom file (.emacs).
9. #8 + loading any library, even indirectly.

Is there is a good inside/outside boundary to choose? Which kinds of
changes should we allow inside the "Set" club?

Users will want to use commands that let them set user options, and
they will expect to see the resulting status in Customize as "Set".
Unless we can identify specific problems with specific levels
(1,2,3...), I'd say that _any_ change to a user option should be
allowed, because the important thing is the user's intention. Any
single cutoff level is a poor measure of intention.

But this is why I wanted a discussion. There may be good reasons,
unidentified so far, for using level 4 or 6 instead of 8 or 9. The
point is to recognize that it is, a priori, _good_ to open up
Customize to the rest of Emacs as much as possible - that is, up to a
level where we see problems with such openness.

IOW, a priori, any way to change a user option is a valid way for a
user to change an option. And all valid ways for a user to change an
option should be reflected in Customize as setting the option. That's
my argument.

The caveat here is that in order to open things up for users we also
need to close them down for Lisp code (and built-in C code -
e.g. `baud-rate') that is inappropriate. User options should not be
trounced indiscrimately by libraries - they are _user_ options. _If_
there is a real need for some library to change a user option (as
opposed to defining a standard value for the option), then maybe that
case should be dealt with specially, or maybe it signals a general
need to stop at level 6 (or whatever) - a fundamental problem.
    
        The idea behind limiting Customize admission to only
        interactive functions I think hints at the proper criterion,
        but it does not hit it square on. The idea of user Helen
        "setting" an option really comes down to this: user
        _intention_. That is not a criterion we can test by program,
        but it is the only reasonable criterion, IMO.
    
    I agree; I think that's exactly what it means to me.  Users set
    variables by running programs.  The difference is that in one case
    the intention to set variable X is embodied in the program code;
    in the other, the progranm could set any variable, but the user
    chooses to set X.

I think we agree, IIUC. Users can use programs to set options. That
doesn't mean that any arbitrary program _should_ set user options. As
someone said earlier, the purpose of the library should explicitly
include and document setting an option.

I hope and expect, actually, that (beyond the bugs) few libraries will
ever need to change user options, at least not by simply being loaded.
Most will a) define standard values for options or b) provide commands
to set options or c) behave themselves properly by modifying only
non-user-option variables.

I should make a distinction here - there are libraries and libraries.
There are libraries that are intended for use by multiple users, and
there are libraries that are essentially custom files or personal
extensions thereof. Multi-user libraries are the ones that should not
step on user options. Personal libraries are made to set user options.

Some libraries (some of mine, for instance) start out life as a
personal library and end up being shared to some extent by others. In
some cases, user-option settings in a library shared this way are not
bothersome - if the person using the library has the same environment
and wants to do the same thing, for instance. This is essentially just
cutting and pasting code.

In general, however, a library destined for multiple users should not
contain user-option settings - that is bad form. I think it would be
good to mention this difference in library kind (purpose, really) in
the "good style" Elisp doc, along with a set of etiquette guidelines
(TBD) for real multi-user libraries.

This difference in library kind (purpose) can be reflected in the
treatment by Customize. It is normal for .emacs and personal libraries
loaded at startup to set user options. Those option values should not
then be considered "changed outside" or even "set" in Customize. They
should be considered as the normal, usual, baseline values for that
user.  That is, they should constitute the "unchanged" zeros of the
Customize world. See my separate "New tack" email for more on this
subject.

What about loading a personal library that sets user options _after_
startup? If I loan you such a file, and you want to try it but are not
sure that you want to add it as is to your startup file, you can load
it after startup and test drive it. If loading it sets user options
(e.g. setq), then those options should be marked "Set" in Customize.
You can then choose to add the library to your startup or perhaps
change a few settings and save some settings to your custom file.

Etiquette guidelines and setting a good example in the standard code
are the proper way to deal with the issue of libraries setting user
options. Although I expect that few multi-user libraries will really
need to set user options, especially during loading, I don't think a
rule (beyond etiquette) should be established against libraries
setting user options. That is, I vote for level #9 or #omega or
whatever: loading any library that changes user options "sets" those
options in Customize.

Unless we can identify problems with such an approach, of course.

Luc's investigations about hooks and other variable-length list
variables are right-on in this regard. That's the kind of thing to
examine: Just which kinds of program changes to user options are
appropriate? Can we find a cleaner way of dealing with hooks, which
leaves the "user" part of user hooks inviolate?

Trying to fix the pbs in the standard libraries will bring about a
better understanding and possibly new implementation techniques, which
can then be reflected in guidelines for developers of other libraries.
    
        A user option is a user option, and it should be totally under
        _user_ control. Proper functioning of a user hook, for
        instance, should not depend on any properties of the hook
        value, such as additional hook functions, that the user does
        not control or is not aware of. If that means that we have to
        split lots of user hooks in two (user; system) or otherwise
        bend over backward to avoid stepping on user settings, then so
        be it - we'll just have to bite the bullet.
    
    That would be a big price to pay.  Even if the other alternative
    is imperfect, it may still be better.

The price might depend on how it's done. You mentioned (albeit in
Customize only):

    treating a single list as if it were the combination of too list
    values, one to be edited through Custom and one to be updated by
    programs.

Or one to be defined/changed by users and one to be defined/changed by
Emacs internals.

If we want to do things to a hook by program, but we don't want to
consider the complete result to be the `standard' value for some
reason, and we also want to keep user stuff inviolable (no changes
behind the back), then one way or the other we will end up with two
parts of the same hook or two different hooks or two different ways to
access the hook or some other way to keep the two separate.

Some inappropriate user-option changes will, I expect, end up being
fixed by just using standard-value definitions instead - perhaps for
part of a hook, for instance.

        It would be useful to separate these two issues even if the
        problems raised did not exist already and were not, therefore,
        unrelated to the proposed UI change.
    
    Maybe-- but we shouldn't change any of these things now, so maybe
    we can just as well consider them both together as part of a
    redesign of the Custom user interface.

100% agreement that we should consider them together. But we shouldn't
get them confused. It is not because there are bugs, which cause lots
of inappropriate "changed-outside" indications, that in a clean system
most outside changes would be inappropriate. Appropriate outside
changes should not be treated differently from inside changes - there
are many ways for a user to set a user option.