RE: single-key-description no good for Japanese and Chinese chars

[Drew Adams]

    > Except for these "invalid" keys, you can use `read-kbd-macro' and call
    > `self-insert-command' on a key from `map-keymaps' to insert it (after
    > setting `last-command-char').

    The fact that it works in most other cases is just accidental.

I see. Then I'd like to identify those cases where it does work, in Lisp.
How can I do that?

I tried to follow the C code for self-insert-command, which starts by
throwing an error if last-command-char is not a character, but I admit that
I don't follow it well.

Also, if self-insert-command is not necessarily intended to "Insert the
character you type", as the doc string says, then maybe that could be
explained a bit more somewhere?

    >     I see no relation between self-insert-command and the other two.

    > See above. I think that Handa explained that you cannot use
    > `self-insert-command' to insert such keys,

    You can't "insert" it, no matter what: it's not a character.

Got it.

    > And I would like to be able to pick up a key from `map-keymap' and
    > execute its command (binding).

    OK, so that's your motivation.  It would help to say it upfront.

I did, at least twice:

      1. I'm mapping over keymaps with `map-keymap' (using a function
         that applies `single-key-description' to the key parts). This
         includes keys that are bound to `self-insert-command'.

         I then convert back the output of `single-key-description',
         using `read-kbd-macro', to get back the key. (These two
         inverse operations are done in two different contexts.)

      2. I just map over keymaps, picking up the keys and their
         bindings. For the keys, I apply `single-key-description'.
         Later, I also need to be able to apply `read-kbd-macro' to
         the result of `single-key-description' to get back the key.

    Now, if you tell us the larger reason why you want to do this,
    we may be able to help you figure out how to do it.

That would be great - thanks for the offer. I already explained the reason,
and, after Handa explained how things work, I understood that what I would
have liked to do for multi-byte chars is not possible - at least not
directly. Anyway, I'll repeat what I said, in case someone has another
suggestion:

      I'm a bit disappointed that I won't be able to use these keys
      with my code, BTW.  I was hoping that this would be a simple bug,
      and these keys (which I thought corresponded to individual chars)
      could just be given unique names.

      That would have enabled my code to provide an alternative,
      poor-man's input method for all languages (as a side effect). My
      code lets you complete against key description-plus-command
      combinations, and for keys bound to `self-insert-command', this
      lets you input the characters by picking them through completion
      (mouse-2, or cycle through them and hit RET), whether or not
      your keyboard can create them. You can either type part or all
      of the key or command name (substring/regexp matching), or you
      can use `M-q' to "quote" keys you hit on the keyboard, to get
      their descriptions in the minibuffer to match the completion
      candidates. (The completion candidates look like this:
      "w  =  self-insert-command".) This works fine for all but the
      "invalid" keys that each correspond to multiple chars - those
      whose descriptions begin "Character set ...".

I was wrong about the last sentence. As you point out below, it also does
not work for keys like [t] and [remap foo]. I'm not sure how to characterize
the keys for which it works, and I'm not sure how to check for only those
keys (i.e. exclude the others) in Lisp - see below.

      I don't know if something along these lines might be adapted to
      use with such languages - perhaps do what I do to first get a
      "generic char", and then somehow complete against the chars in
      that character group? Any ideas on that or any interest in it?
      My approach couldn't replace input methods, obviously, but it
      might be handy when you don't want to spend the time to learn
      an input method for a few characters, and it could perhaps be
      helpful to people who are handicapped (e.g. find it difficult
      to deal with whole keyboards, but can use a mouse). You can see
      all the possible matching characters, and just click the ones
      you want. Pick first a character group (whatever that
      corresponds to), and then pick a character within the group -
      all via completion. Anyway, if you do happen to be interested:
      http://www.emacswiki.org/cgi-bin/wiki/Icicles_-_Key_Completion.

    > I would expect command `self-command-insert' to insert the
    > key, and that doesn't work for the "invalid" keys.

    You can't insert a key, you can only insert a char.  Some keys
    can be turned (more or less trivially) into chars, but not all.

I understand. I wrote "char (key)" and "key (char)" in most of the places
where I spoke of this.  I was thinking in terms of what you call the "keys
[that] can be turned (more or less trivially) into chars." (Initially, I
thought that this was true of all keys bound to `self-insert-command' - I
didn't know about the "invalid" or non-trivial keys.)

    >> A program might well expect `self-insert-command' to do what it says
    >> straightforwardly: insert the key as a character.
    >     What makes you think it doesn't do exactly that?
    > See above, and Handa's comment that "it doesn't work, it
    > signals an error."

    It can't do it because it's an impossible request.  self-insert-command
    can't be blamed for it, and no other implementation will solve
    your problem.

[Note: First you ask "What makes you think it doesn't do exactly that?", and
then you say it obviously could never do that. ;-)]

My suggestion was to use `self-insert-command' only for "keys that can be
turned (more or less trivially) into chars", and to use another function,
say `self-insert-invalid-key' or `self-insert-non-trivial-key', which treats
the "invalid" keys that Handa referred to and any other keys that cannot be
turned trivially into chars (all of which today are bound to
`self-insert-command'). This function would be identical to today's
`self-insert-command', but it would only be called for the non-trivial keys.

Wouldn't that work? All programs could then expect that any keys bound to
`self-insert-command' would work in the way I explained - no need to test
each key. `self-insert-command' would be reserved for the straightforward,
trivial case, so you could count on keys bound to it being in that case.

That wouldn't provide a way to insert the multi-byte chars (or use the
non-trivial keys) via `self-insert-command', but it would prevent them from
getting in the way there. It would mean that you can always call
`self-insert-command' to insert the char whose associated key is bound to
`self-insert-command'.

Anyway, that was just a suggestion. Counter arguments expressed were all
along the lines of "It's OK to do the non-trivial stuff too in
`self-insert-command'." I can't argue with that. I only suggested separating
the trivial case from the non-trivial by binding them to two different
commands (even if the commands did the same thing). That would make it easy
to tell which case you were dealing with - just look at the binding.

    > And I tried it, and it didn't work for me: get the key via
    > `map-keymap', get its `single-key-description', get
    > `read-kbd-macro' of that, set `last-command-char' to that,
    > and call `self-insert-command'.
    > OK for all keys except these "invalid" keys. Seems like a
    > reasonable operation to expect you could do, no?

    No.  Try your code with the following map:

         (let ((map (make-sparse-keymap)))
           (define-key map [t] 'self-insert-command)
           (define-key map [remap tab-to-tab-stop] 'self-insert-command)
           ...)

    and you'll see other instances of the similar problem:
    self-insert-command may be bound to chars in indirect ways.
    Generic chars are just one such case among others.

I see now. How can I test for (all of) these cases? How can I test a key to
see if it is one "that can be turned (more or less trivially) into" a char?
That would let me weed out all other (non-trivial) keys. Currently, I'm
weeding out only the multi-byte stuff, by testing the
`single-key-description's (which is roundabout).

I don't know how to weed out `remap' and such. `remap' gets treated in my
code like a prefix key, because it is bound to a keymap (at least the manual
defines a prefix key that way). I've been using `keymapp' to test for being
a prefix key, but that returns t for stuff like `remap' too.

Then, when I go to use the prefix key `remap', via (accessible-keymaps
(current-global-map) "remap"), I get nil. This means that my code first
displays `remap' to users as a possible prefix key, but then if a user
chooses that prefix I have to raise an error saying "No keys for prefix
`remap'.

Any suggestions about this are welcome. Here's the code:

  (defun icicle-complete-keys ()
    "Complete a key sequence for the currently invoked prefix key."
    (interactive)
    (let ((icicle-show-Completions-initially-flag t)
          (enable-recursive-minibuffers t))
      (icicle-complete-keys-1
        (let ((this-key (this-command-keys)))
          (substring this-key 0 (1- (length this-key)))))))

  (defun icicle-complete-keys-1 (prefix)
    "Complete a key sequence for prefix key PREFIX."
    (let* ((icicle-must-not-match-regexp ; This weeds out generic chars.
             "^Character set .*=  self-insert-command")
           (keys+cmds (icicle-keys+cmds-w-prefix prefix)))
      (unless keys+cmds (error "No keys for prefix `%s'" prefix))
      (let ((choice (completing-read
                     (concat "Complete keys " (key-description prefix) ": ")
                     keys+cmds nil t)))
        (string-match "\\(.+\\)  =  \\(.+\\)" choice)
        (let ((key (save-match-data
                     (read-kbd-macro
                      (substring choice (match-beginning 1) (match-end 1))
                      t)))
              (cmd (substring choice (match-beginning 2) (match-end 2))))
          (if (string= "..." cmd)
              ;; Complete keys with KEY as prefix.
              (icicle-complete-keys-1 (concat prefix key))
            (setq cmd (intern cmd))
            (when (eq 'self-insert-command cmd)
              (setq last-command-char (aref key 0)))
            (call-interactively cmd))))))

  (defun icicle-keys+cmds-w-prefix (prefix)
    "Return alist of keys and bindings for prefix key PREFIX."
    (let* ((keys+maps (accessible-keymaps (current-global-map) prefix))
           (map (cdar keys+maps))
           (keys+cmds nil))
      (when keys+maps
        (map-keymap (lambda (event binding)
                      (push `((,(single-key-description event 'no-angles)
                               ,(if (keymapp binding)
                                    "..."       ; Prefix key
                                    (format "%s" binding))))
                            keys+cmds))
                    map))
      keys+cmds))