Re: [patch] make electric-pair-mode smarter/more useful

[João Távora]

Stefan Monnier <address@hidden> writes:
> Overall, the integration looks very good, indeed.
>

Thanks. So I went ahead and considerably the implementation along the
lines suggested. I'm quite happy with it having implemented most of
autopair's core features in a way that I still think is tightly
integrating with the existing electric.el backend.

I'd be happy to get comments and perform any adjustments.

Skip to to patch at the end of the mail or read some key points:

- pairing skipping of parens/quotes is balanced in both code, strings
  and comments.

- electric backspacing was implemented per your remapping suggestion.

- autowrapping was enhanced to also happen on closers.

- there are just over 500 tests in
  tests/automated/electric-pair-tests.el. Obviously I wrote some 30 or
  40 and the rest are variations of in different major modes (elisp, c++
  and ruby-mode), in strings and comments. Some js-mode tests are also
  there for its use of `electric-layout-rules'.

- `electric-pair-pairs` now defaults to nil.

  See its updated docstring for the reasoning and the new (akwardly
  named) variables `electric-pair-non-code-pairs' and
  `electric-pair-non-code-syntax-table' for slightly better, more
  flexible alternatives to it, in my opinion.

  Maybe it's built as overkill, but I could not find a simpler way to
  get autopair.el's backtick-and-quote pairing for elisp comments and
  strings...other than customizing electric-inhibit/skip callbacks in my
  .emacs, of course.

  But I do feel this also belongs electric.el, and was a common request
  in autopair.el, which has even more flexibility for this (it has
  :code, :string, :comments and :everywhere modifiers).

- `electric-pair--pair-of' and `electric-pair-syntax' have merged, as
  you suggested, into `electric-pair-syntax-info'. This is also where
  the new variables above come into play.

- there is a new `electric--sort-post-self-insertion-hook' function
  called whenever a hook is added to `post-self-insert-hook'. As you had
  already noted in the "FIXME: ugly!" note (now removed), the `append'
  arg to `add-hook' doesn't cut it.

  I read the other FIXME notes but didn't understand them so well. Can
  you provide examples of the conflicts between `electric-indent-mode'
  and other `electric-modes' that you mention there?

- I simplififed the previous `electric-pair--up-list' function and
  renamed it `electric-pair--balance-info'. But I still couldn't make it
  use `up-list' or get rid of some `forward-sexp'. `up-list' can
  probably be done with enough care, but I think replacing
  `forward-sexp' with `syntax-ppss' is only possible in the "pair", not
  the "skip" case. And only when outside comments or strings.

  I do agree that too many `forward-sexp' can be hazardous (I had some
  bad reports in autopair.el), but i recall that it was only when the
  number of calls is proportional to buffer size.

  In this implementation, I think the the number of `forward-sexp''s is
  at most proportional to the nesting depth, which is less
  dangerous.

  But anyway if you or someone more knowlegeable in emacs's syntax
  parsing engine can figure out a way to make this and still pass all
  the tests, all the better.

- some helper functions might be reinventing the wheel, such as
  `electric-pair--looking-at-mismatched-string-p' and
  `electric-pair--inside-comment-string'.

- I'm also trying my luck and changed lisp-mode.el defaults to
  accomodate some of my preferred defaults in variables
  `electric-pair-skip-whitespace' and `electric-pair-non-code-pairs').

>> +(defun electric-pair--pair-of (char)
> Hmmm... the existing code already has such a functionality.  Can you try
> and use your new function in that code, or somehow merge the two?

Done, see above.

>> +(defun electric-pair--up-list (&optional n)
> Could you try and use up-list, instead?

I tried, I failed. See above.

> Also, you can find the START of all enclosing lists in (nth
> 9 (syntax-ppss)), which seems like it might be helpful here.

I tried, I removed one forward-sexp in the "pair" case. See inline
comments.


diff --git a/lisp/electric.el b/lisp/electric.el
index 91b99b4..b227e3d 100644
--- a/lisp/electric.el
+++ b/lisp/electric.el
@@ -187,6 +187,27 @@ Returns nil when we can't find this char."
                            (eq (char-before) last-command-event)))))
       pos)))

+(defun electric--sort-post-self-insertion-hook ()
+  "Ensure order of electric functions in `post-self-insertion-hook'.
+
+Hooks in this variable interact in non-trivial ways, so a
+relative order must be maintained within it."
+  (let ((relative-order '(electric-pair-post-self-insert-function
+                          electric-layout-post-self-insert-function
+                          electric-indent-post-self-insert-function
+                          blink-paren-post-self-insert-function)))
+    (setq post-self-insert-hook
+          (sort post-self-insert-hook
+                #'(lambda (fn1 fn2)
+                    (let ((fn1-tail (memq fn1 relative-order))
+                          (fn2-tail (memq fn2 relative-order)))
+                      (cond ((and fn1-tail fn2-tail)
+                             (> (length fn1-tail)
+                                (length fn2-tail)))
+                            (fn1-tail t)
+                            (fn2-tail nil)
+                            (t nil))))))))
+
 ;;; Electric indentation.

 ;; Autoloading variables is generally undesirable, but major modes
@@ -295,20 +316,9 @@ insert a character from `electric-indent-chars'."
                      #'electric-indent-post-self-insert-function))
     (when (eq (lookup-key global-map [?\C-j]) 'newline-and-indent)
       (define-key global-map [?\C-j] 'electric-indent-just-newline))
-    ;; post-self-insert-hooks interact in non-trivial ways.
-    ;; It turns out that electric-indent-mode generally works better if run
-    ;; late, but still before blink-paren.
     (add-hook 'post-self-insert-hook
-              #'electric-indent-post-self-insert-function
-              'append)
-    ;; FIXME: Ugly!
-    (let ((bp (memq #'blink-paren-post-self-insert-function
-                    (default-value 'post-self-insert-hook))))
-      (when (memq #'electric-indent-post-self-insert-function bp)
-        (setcar bp #'electric-indent-post-self-insert-function)
-        (setcdr bp (cons #'blink-paren-post-self-insert-function
-                         (delq #'electric-indent-post-self-insert-function
-                               (cdr bp))))))))
+              #'electric-indent-post-self-insert-function)
+    (electric--sort-post-self-insertion-hook)))

 ;;;###autoload
 (define-minor-mode electric-indent-local-mode
@@ -326,33 +336,121 @@ insert a character from `electric-indent-chars'."
 ;;; Electric pairing.

 (defcustom electric-pair-pairs
-  '((?\" . ?\"))
-  "Alist of pairs that should be used regardless of major mode."
+  '()
+  "Alist of pairs that should be used regardless of major mode.
+
+Pairs of delimiters in this list cannot be balanced automatically, so
+before adding to this variable, consider modifying your mode's syntax
+table.
+
+See also the variable `electric-pair-non-code-pairs'."
   :version "24.1"
   :type '(repeat (cons character character)))

-(defcustom electric-pair-skip-self t
+(defcustom electric-pair-non-code-pairs
+  '()
+  "Alist of pairs that should be used only when in strings or comments.
+
+Pairs of delimiters in this list cannot be balanced automatically, so
+before adding to this variable, consider modifying the \(buffer-local)
+value of the variable `electric-pair-non-code-syntax-table'."
+  :version "24.4"
+  :type '(repeat (cons character character)))
+
+(make-variable-buffer-local 'electric-pair-non-code-pairs)
+
+(defcustom electric-pair-skip-self #'electric-pair-skip-if-helps-balance
   "If non-nil, skip char instead of inserting a second closing paren.
+
 When inserting a closing paren character right before the same character,
 just skip that character instead, so that hitting ( followed by ) results
 in \"()\" rather than \"())\".
-This can be convenient for people who find it easier to hit ) than C-f."
+
+This can be convenient for people who find it easier to hit ) than C-f.
+
+Can also be a function of one argument (the closer char just
+inserted), in which case that function's return value is
+considered instead."
   :version "24.1"
-  :type 'boolean)
+  :type '(choice
+          (const :tag "Always skip" t)
+          (const :tag "Never skip" nil)
+          (const :tag "Help balance" electric-pair-skip-if-helps-balance)
+          function))

 (defcustom electric-pair-inhibit-predicate
-  #'electric-pair-default-inhibit
+  #'electric-pair-inhibit-if-helps-balance
   "Predicate to prevent insertion of a matching pair.
+
 The function is called with a single char (the opening char just inserted).
 If it returns non-nil, then `electric-pair-mode' will not insert a matching
 closer."
   :version "24.4"
   :type '(choice
-          (const :tag "Default" electric-pair-default-inhibit)
+          (const :tag "Conservative" electric-pair-conservative-inhibit)
+          (const :tag "Help balance" electric-pair-inhibit-if-helps-balance)
           (const :tag "Always pair" ignore)
           function))

-(defun electric-pair-default-inhibit (char)
+(defcustom electric-pair-delete-adjacent-pairs t
+  "If non-nil, backspacing an open paren also deletes ajacent closer.
+
+Can also be a function of no arguments, in which case that function's
+return value is considered instead."
+  :version "24.4"
+  :type '(choice
+          (const :tag "Yes" t)
+          (const :tag "No" nil)
+          function))
+
+(defcustom electric-pair-skip-whitespace t
+  "If non-nil skip whitespace when skipping over closing parens.
+
+Can also be a function of no arguments, in which case that function's
+return value is considered instead."
+  :version "24.4"
+  :type '(choice
+          (const :tag "Yes, jump over whitespace" t)
+          (const :tag "Yes, and delete whitespace" 'chomp)
+          (const :tag "No, no whitespace skipping" nil)
+          function))
+
+(defvar electric-pair-non-code-syntax-table prog-mode-syntax-table
+  "Syntax table used when pairing inside comments and strings.
+
+`electric-pair-mode' considers this syntax table only when point in inside
+quotes or comments, and only after examining `electric-pair-pairs'.")
+
+(defun electric-pair-backward-delete-char (n &optional killflag untabify)
+  "Delete characters backward, and maybe also two adjacent paired delimiters.
+
+Remaining behaviour is given by `backward-delete-char' or, if
+UNTABIFY is non-nil, `backward-delete-char-untabify'."
+  (interactive "*p\nP")
+  (let* ((prev (char-before))
+         (next (char-after))
+         (syntax-info (electric-pair-syntax-info prev))
+         (syntax (car syntax-info))
+         (pair (cadr syntax-info)))
+    (when (and (if (functionp electric-pair-delete-adjacent-pairs)
+                   (funcall electric-pair-delete-adjacent-pairs)
+                 electric-pair-delete-adjacent-pairs)
+               next
+               (memq syntax '(?\( ?\" ?\$))
+               (eq pair next))
+      (delete-char 1 killflag))
+    (if untabify
+        (backward-delete-char-untabify n killflag)
+        (backward-delete-char n killflag))))
+
+(defun electric-pair-backward-delete-char-untabify (n &optional killflag)
+  "Delete characters backward, and maybe also two adjacent paired delimiters.
+
+Remaining behaviour is given by `backward-delete-char-untabify'."
+  (interactive "*p\nP")
+  (electric-pair-backward-delete-char n killflag t))
+
+(defun electric-pair-conservative-inhibit (char)
   (or
    ;; I find it more often preferable not to pair when the
    ;; same char is next.
@@ -363,14 +461,40 @@ closer."
    ;; I also find it often preferable not to pair next to a word.
    (eq (char-syntax (following-char)) ?w)))

-(defun electric-pair-syntax (command-event)
-  (let ((x (assq command-event electric-pair-pairs)))
+(defun electric-pair-syntax-info (command-event)
+  "Calculate a list (SYNTAX PAIR WHERE).
+
+SYNTAX is COMMAND-EVENT's syntax character, PAIR is its pair and WHERE is
+either a syntax table or `t', meaning \"everywhere\""
+  (let* ((pre-comment-or-string-p (save-excursion
+                                    (nth 8 (syntax-ppss (1- (point))))))
+         (post-comment-or-string-p (nth 8 (syntax-ppss)))
+         (comment-or-string-p
+          (and post-comment-or-string-p
+               pre-comment-or-string-p))
+         (direct (or (assq command-event electric-pair-pairs)
+                     (and comment-or-string-p
+                          (assq command-event electric-pair-non-code-pairs))))
+         (reverse (or (rassq command-event electric-pair-pairs)
+                      (and comment-or-string-p
+                           (rassq command-event 
electric-pair-non-code-pairs))))
+         (table (if comment-or-string-p
+                    electric-pair-non-code-syntax-table
+                  (syntax-table))))
     (cond
-     (x (if (eq (car x) (cdr x)) ?\" ?\())
-     ((rassq command-event electric-pair-pairs) ?\))
-     ((nth 8 (syntax-ppss))
-      (with-syntax-table text-mode-syntax-table (char-syntax command-event)))
-     (t (char-syntax command-event)))))
+     (direct (if (eq (car direct) (cdr direct))
+                 (list ?\" command-event t)
+               (list ?\( (cdr direct) t)))
+     (reverse (list ?\) (car reverse) t))
+     (t
+      (with-syntax-table table
+        (list (char-syntax command-event)
+              (or (matching-paren command-event)
+                  command-event)
+              table))))))
+
+(defun electric-pair--pair-of (char)
+  (cadr (electric-pair-syntax-info char)))

 (defun electric-pair--insert (char)
   (let ((last-command-event char)
@@ -378,56 +502,235 @@ closer."
        (electric-pair-mode nil))
     (self-insert-command 1)))

+(defun electric-pair--matched-p (here direction)
+  "Tell if the delimiter at point HERE is perfectly matched.
+
+With positive DIRECTION consider the delimiter after HERE and
+search forward, otherwise consider the delimiter is just before
+HERE and search backward."
+  ;; FIXME: Ideally no `forward-sexp'eeing should take place here, but
+  ;; we can only avoid it to find out if a sexp before point is
+  ;; matched. It won't work for one after point or one inside
+  ;; comments.
+  ;;
+  ;; We could also use `show-paren-data-function' here, it seems to
+  ;; always provide reliable results.
+  ;;
+  (cond ((> direction 0)
+         (condition-case move-err
+             (save-excursion
+               (forward-sexp 1)
+               (eq (char-after here)
+                   (electric-pair--pair-of (char-before (point)))))
+           (scan-error nil)))
+        ((nth 8 (syntax-ppss))
+         (condition-case move-err
+             (save-excursion
+               (forward-sexp -1)
+               (eq (char-before here)
+                   (electric-pair--pair-of (char-after (point)))))
+           (scan-error nil)))
+        (t
+         ;; we can use some `syntax-ppss' in this case,
+         ;; no need to `forward-sexp' back
+         (save-excursion
+           (goto-char (1- (point)))
+           (let ((start (car (nth 9 (syntax-ppss)))))
+             (eq (char-before here)
+                 (electric-pair--pair-of (char-after start))))))))
+
+(defun electric-pair--balance-info (n)
+  "Examine lists forward or backward according to N's sign.
+
+Return a cons of two descritions (MATCHED . PAIR) for the
+innermost and outermost lists that enclose point. The outermost
+list enclosing point is either the first top-level or mismatched
+list found by uplisting.
+
+If the outermost list is matched, don't rely on its PAIR. If
+point is not enclosed by any lists, return ((T) (T))."
+  (save-excursion
+    (let (innermost outermost)
+      (while (not outermost)
+        (condition-case err
+            (progn
+              (scan-sexps (point) (if (> n 0)
+                                      (point-max)
+                                    (- (point-max))))
+              (setq outermost (list t))
+              (unless innermost
+                (setq innermost (list t))))
+          (scan-error
+           (goto-char (nth 3 err))
+           (let ((matched (electric-pair--matched-p (nth 3 err) (- n)))
+                 (actual-pair (if (> n 0)
+                                  (char-before (point))
+                                (char-after (point)))))
+             (unless innermost
+               (setq innermost (cons matched actual-pair)))
+             (unless matched
+               (setq outermost (cons matched actual-pair)))))))
+      (cons innermost outermost))))
+
+(defun electric-pair--looking-at-mismatched-string-p ()
+  "Say if the nearest string started after point is mismatched."
+  (save-excursion
+    (skip-syntax-forward "^?\"")
+    (while (not (zerop (% (save-excursion (skip-syntax-backward "\\")) 2)))
+      (unless (eobp)
+        (forward-char 1)
+        (skip-syntax-forward "^?\"")))
+    (and (not (eobp))
+         (condition-case err
+             (progn (forward-sexp) nil)
+           (scan-error
+            t)))))
+
+(defun electric-pair--inside-comment-string ()
+  "When inside a comment, say if point is inside a string."
+  ;; FIXME: ugly/naive
+  (save-excursion
+    (save-restriction
+      (narrow-to-region (nth 8 (syntax-ppss)) (point))
+      (let ((non-escaped-quotes 0))
+        (while (not (bobp))
+          (skip-syntax-backward "^?\"")
+          (unless (bobp)
+            (backward-char))
+          (when (and (not (bobp))
+                     (zerop (% (save-excursion (skip-syntax-backward "\\")) 
2)))
+            (setq non-escaped-quotes (1+ non-escaped-quotes))))
+        (not (zerop (% non-escaped-quotes 2)))))))
+
+(defun electric-pair-inhibit-if-helps-balance (char)
+  "Return non-nil if auto-pairing of CHAR would hurt parentheses' balance.
+
+Works by first removing the character from the buffer, then doing
+some list calculations, finally restoring the situation as if nothing
+happened."
+  (let* ((syntax-info (electric-pair-syntax-info char))
+         (syntax (car syntax-info))
+         (pair (cadr syntax-info)))
+    (unwind-protect
+        (progn
+          (delete-char -1)
+          (cond ((eq ?\( syntax)
+                 (let* ((pair-data (electric-pair--balance-info 1))
+                        (innermost (car pair-data))
+                        (outermost (cdr pair-data)))
+                   (cond ((car outermost)
+                          nil)
+                         ((not (car innermost))
+                          (eq (cdr outermost) pair))
+                         (t
+                          t))))
+                ((eq syntax ?\")
+                 (let ((string-start (nth 3 (syntax-ppss))))
+                   (or (eq string-start t)
+                       (eq string-start char)
+                       (electric-pair--looking-at-mismatched-string-p))))))
+      (insert-char char))))
+
+(defun electric-pair-skip-if-helps-balance (char)
+  "Return non-nil if skipping CHAR would benefit parentheses' balance.
+
+Works by first removing the character from the buffer, then doing
+some list calculations, finally restoring the situation as if nothing
+happened."
+  (let* ((syntax-info (electric-pair-syntax-info char))
+         (syntax (car syntax-info))
+         (pair (cadr syntax-info)))
+    (unwind-protect
+        (progn
+          (delete-char -1)
+          (cond ((eq syntax ?\))
+                 (let* ((pair-data (electric-pair--balance-info -1))
+                        (innermost (car pair-data))
+                        (outermost (cdr pair-data)))
+                   (and
+                    (cond ((car outermost)
+                           (car innermost))
+                          ((car innermost)
+                           (not (eq (cdr outermost) pair)))))))
+                ((eq syntax ?\")
+                 (let ((string-start (nth 3 (syntax-ppss))))
+                   (or (eq string-start t)
+                       (eq string-start char)
+                       (not (electric-pair--looking-at-mismatched-string-p))
+                       (and (nth 8 (syntax-ppss))
+                            (electric-pair--inside-comment-string)))))))
+      (insert-char char))))
+
 (defun electric-pair-post-self-insert-function ()
   (let* ((pos (and electric-pair-mode (electric--after-char-pos)))
-        (syntax (and pos (electric-pair-syntax last-command-event)))
-         (closer (if (eq syntax ?\()
-                     (cdr (or (assq last-command-event electric-pair-pairs)
-                              (aref (syntax-table) last-command-event)))
-                   last-command-event)))
+         (syntax-info (and pos (electric-pair-syntax-info last-command-event)))
+         (syntax (car syntax-info))
+         (pair (cadr syntax-info))
+         (table (or (caddr syntax-info) (syntax-table))))
     (cond
      ((null pos) nil)
      ;; Wrap a pair around the active region.
-     ((and (memq syntax '(?\( ?\" ?\$)) (use-region-p))
+     ;;
+     ((and (memq syntax '(?\( ?\) ?\" ?\$)) (use-region-p))
       ;; FIXME: To do this right, we'd need a post-self-insert-function
       ;; so we could add-function around it and insert the closer after
       ;; all the rest of the hook has run.
-      (if (>= (mark) (point))
-         (goto-char (mark))
-       ;; We already inserted the open-paren but at the end of the
-       ;; region, so we have to remove it and start over.
-       (delete-region (1- pos) (point))
-       (save-excursion
-          (goto-char (mark))
-          (electric-pair--insert last-command-event)))
-      ;; Since we're right after the closer now, we could tell the rest of
-      ;; post-self-insert-hook that we inserted `closer', but then we'd get
-      ;; blink-paren to kick in, which is annoying.
-      ;;(setq last-command-event closer)
-      (insert closer))
+      (if (or (eq syntax ?\")
+              (and (eq syntax ?\))
+                   (>= (point) (mark)))
+              (and (not (eq syntax ?\)))
+                   (>= (mark) (point))))
+          (save-excursion
+            (goto-char (mark))
+            (electric-pair--insert pair))
+        (delete-region pos (1- pos))
+        (electric-pair--insert pair)
+        (goto-char (mark))
+        (electric-pair--insert last-command-event)))
      ;; Backslash-escaped: no pairing, no skipping.
      ((save-excursion
         (goto-char (1- pos))
         (not (zerop (% (skip-syntax-backward "\\") 2))))
       nil)
+     ;; Insert matching pair.
+     ((and (memq syntax `(?\( ?\" ?\$))
+           (not overwrite-mode)
+           (or (eq table t)
+               (with-syntax-table table
+                 (not (funcall electric-pair-inhibit-predicate 
last-command-event)))))
+      (save-excursion (electric-pair--insert pair)))
      ;; Skip self.
      ((and (memq syntax '(?\) ?\" ?\$))
-           electric-pair-skip-self
-           (eq (char-after pos) last-command-event))
-      ;; This is too late: rather than insert&delete we'd want to only skip (or
-      ;; insert in overwrite mode).  The difference is in what goes in the
-      ;; undo-log and in the intermediate state which might be visible to other
-      ;; post-self-insert-hook.  We'll just have to live with it for now.
-      (delete-char 1))
-     ;; Insert matching pair.
-     ((not (or (not (memq syntax `(?\( ?\" ?\$)))
-               overwrite-mode
-               (funcall electric-pair-inhibit-predicate last-command-event)))
-      (save-excursion (electric-pair--insert closer))))))
+           (or (eq table t)
+               (if (functionp electric-pair-skip-self)
+                   (with-syntax-table table
+                     (funcall electric-pair-skip-self last-command-event))
+                 electric-pair-skip-self)))
+      (let ((original-point (point))
+            (skip-info (if (functionp electric-pair-skip-whitespace)
+                           (funcall electric-pair-skip-whitespace)
+                         electric-pair-skip-whitespace)))
+        (when skip-info (skip-chars-forward "\t\s\n"))
+        (if (eq (char-after (if skip-info
+                                (point)
+                              pos))
+                last-command-event)
+            ;; This is too late: rather than insert&delete we'd want
+            ;; to only skip (or insert in overwrite mode).  The
+            ;; difference is in what goes in the undo-log and in the
+            ;; intermediate state which might be visible to other
+            ;; post-self-insert-hook.  We'll just have to live with it
+            ;; for now.
+            (if (eq skip-info 'chomp)
+                (delete-region original-point (1+ (point)))
+              (delete-region (1- pos) pos)
+              (forward-char))
+          (goto-char original-point)))))))

 (defun electric-pair-will-use-region ()
   (and (use-region-p)
-       (memq (electric-pair-syntax last-command-event) '(?\( ?\" ?\$))))
+       (memq (car (electric-pair-syntax-info last-command-event))
+             '(?\( ?\) ?\" ?\$))))

 ;;;###autoload
 (define-minor-mode electric-pair-mode
@@ -442,10 +745,19 @@ closing parenthesis.  \(Likewise for brackets, etc.)

 See options `electric-pair-pairs' and `electric-pair-skip-self'."
   :global t :group 'electricity
+  :keymap (let ((map (make-sparse-keymap)))
+            (define-key map [remap backward-delete-char-untabify]
+              'electric-pair-backward-delete-char-untabify)
+            (define-key map [remap backward-delete-char]
+              'electric-pair-backward-delete-char)
+            (define-key map [remap delete-backward-char]
+              'electric-pair-backward-delete-char)
+            map)
   (if electric-pair-mode
       (progn
        (add-hook 'post-self-insert-hook
                  #'electric-pair-post-self-insert-function)
+        (electric--sort-post-self-insertion-hook)
        (add-hook 'self-insert-uses-region-functions
                  #'electric-pair-will-use-region))
     (remove-hook 'post-self-insert-hook
@@ -494,11 +806,13 @@ positive, and disable it otherwise.  If called from Lisp, 
enable
 the mode if ARG is omitted or nil.
 The variable `electric-layout-rules' says when and how to insert newlines."
   :global t :group 'electricity
-  (if electric-layout-mode
-      (add-hook 'post-self-insert-hook
-                #'electric-layout-post-self-insert-function)
-    (remove-hook 'post-self-insert-hook
-                 #'electric-layout-post-self-insert-function)))
+  (cond (electric-layout-mode
+         (add-hook 'post-self-insert-hook
+                   #'electric-layout-post-self-insert-function)
+         (electric--sort-post-self-insertion-hook))
+        (t
+         (remove-hook 'post-self-insert-hook
+                      #'electric-layout-post-self-insert-function))))

 (provide 'electric)

diff --git a/lisp/emacs-lisp/lisp-mode.el b/lisp/emacs-lisp/lisp-mode.el
index f4e9b31..8bb74d3 100644
--- a/lisp/emacs-lisp/lisp-mode.el
+++ b/lisp/emacs-lisp/lisp-mode.el
@@ -472,7 +472,11 @@ font-lock keywords will not be case sensitive."
          (font-lock-mark-block-function . mark-defun)
          (font-lock-syntactic-face-function
           . lisp-font-lock-syntactic-face-function)))
-  (setq-local prettify-symbols-alist lisp--prettify-symbols-alist))
+  (setq-local prettify-symbols-alist lisp--prettify-symbols-alist)
+  ; electric
+  (when elisp
+    (setq-local electric-pair-non-code-pairs '((?\` . ?\'))))
+  (setq-local electric-pair-skip-whitespace 'chomp))

 (defun lisp-outline-level ()
   "Lisp mode `outline-level' function."
diff --git a/lisp/simple.el b/lisp/simple.el
index 260c170..c591cee 100644
--- a/lisp/simple.el
+++ b/lisp/simple.el
@@ -607,7 +607,7 @@ In some text modes, where TAB inserts a tab, this command 
indents to the
 column specified by the function `current-left-margin'."
   (interactive "*")
   (delete-horizontal-space t)
-  (newline)
+  (newline 1 (not (or executing-kbd-macro noninteractive)))
   (indent-according-to-mode))

 (defun reindent-then-newline-and-indent ()
diff --git a/test/automated/electric-tests.el b/test/automated/electric-tests.el
new file mode 100644
index 0000000..feaea0a
--- /dev/null
+++ b/test/automated/electric-tests.el
@@ -0,0 +1,436 @@
+;;; electric-tests.el --- tests for electric.el -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2013  João Távora
+
+;; Author: João Távora <address@hidden>
+;; Keywords:
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;;
+
+;;; Code:
+(require 'ert)
+(require 'ert-x)
+(require 'electric)
+(require 'cl-lib)
+
+(defun call-with-saved-electric-modes (fn)
+  (let ((saved-electric (if electric-pair-mode 1 -1))
+        (saved-layout (if electric-layout-mode 1 -1))
+        (saved-indent (if electric-indent-mode 1 -1)))
+    (electric-pair-mode -1)
+    (electric-layout-mode -1)
+    (electric-indent-mode -1)
+    (unwind-protect
+        (funcall fn)
+      (electric-pair-mode saved-electric)
+      (electric-indent-mode saved-indent)
+      (electric-layout-mode saved-layout))))
+
+(defmacro save-electric-modes (&rest body)
+  (declare (indent defun) (debug t))
+  `(call-with-saved-electric-modes #'(lambda () ,@body)))
+
+(defun electric-pair-test-for (fixture where char expected-string
+                                       expected-point mode bindings fixture-fn)
+  (with-temp-buffer
+    (funcall mode)
+    (insert fixture)
+    (save-electric-modes
+     (let ((last-command-event char))
+       (goto-char where)
+       (funcall fixture-fn)
+       (progv
+           (mapcar #'car bindings)
+           (mapcar #'cdr bindings)
+         (self-insert-command 1))))
+    (should (equal (buffer-substring-no-properties (point-min) (point-max))
+                   expected-string))
+    (should (equal (point)
+                   expected-point))))
+
+(eval-when-compile
+  (defun electric-pair-define-test-form (name fixture
+                                              char
+                                              pos
+                                              expected-string
+                                              expected-point
+                                              skip-pair-string
+                                              prefix
+                                              suffix
+                                              extra-desc
+                                              mode
+                                              bindings
+                                              fixture-fn)
+    (let* ((expected-string-and-point
+            (if skip-pair-string
+                (with-temp-buffer
+                  (progv
+                      ;; FIXME: avoid `eval'
+                      (mapcar #'car (eval bindings))
+                      (mapcar #'cdr (eval bindings))
+                    (funcall mode)
+                    (insert fixture)
+                    (goto-char (1+ pos))
+                    (insert char)
+                    (cond ((eq (aref skip-pair-string pos)
+                               ?p)
+                           (insert (electric-pair--pair-of char))
+                           (backward-char 1))
+                          ((eq (aref skip-pair-string pos)
+                               ?s)
+                           (delete-char -1)
+                           (forward-char 1)))
+                    (list
+                     (buffer-substring-no-properties (point-min) (point-max))
+                     (point))))
+              (list expected-string expected-point)))
+           (expected-string (car expected-string-and-point))
+           (expected-point (cadr expected-string-and-point))
+           (fixture (format "%s%s%s" prefix fixture suffix))
+           (expected-string (format "%s%s%s" prefix expected-string suffix))
+           (expected-point (+ (length prefix) expected-point))
+           (pos (+ (length prefix) pos)))
+      `(ert-deftest ,(intern (format "electric-pair-%s-at-point-%s-in-%s%s"
+                                     name
+                                     (1+ pos)
+                                     mode
+                                     extra-desc))
+           ()
+         ,(format "With \"%s\", try input %c at point %d. \
+Should %s \"%s\" and point at %d"
+                  fixture
+                  char
+                  (1+ pos)
+                  (if (string= fixture expected-string)
+                      "stay"
+                    "become")
+                  (replace-regexp-in-string "\n" "\\\\n" expected-string)
+                  expected-point)
+         (electric-pair-test-for ,fixture
+                                 ,(1+ pos)
+                                 ,char
+                                 ,expected-string
+                                 ,expected-point
+                                 ',mode
+                                 ,bindings
+                                 ,fixture-fn)))))
+
+(cl-defmacro define-electric-pair-test
+    (name fixture
+          input
+          &key
+          skip-pair-string
+          expected-string
+          expected-point
+          bindings
+          (modes '(quote (emacs-lisp-mode ruby-mode c++-mode)))
+          (test-in-comments t)
+          (test-in-strings t)
+          (test-in-code t)
+          (fixture-fn #'(lambda ()
+                          (electric-pair-mode 1))))
+  `(progn
+     ,@(cl-loop
+        for mode in (eval modes) ;FIXME: avoid `eval'
+        append
+        (cl-loop
+         for (prefix suffix extra-desc) in
+         (append (if test-in-comments
+                     `((,(with-temp-buffer
+                           (funcall mode)
+                           (insert "z")
+                           (comment-region (point-min) (point-max))
+                           (buffer-substring-no-properties (point-min)
+                                                           (1- (point-max))))
+                        ""
+                        "-in-comments")))
+                 (if test-in-strings
+                     `(("\"" "\"" "-in-strings")))
+                 (if test-in-code
+                     `(("" "" ""))))
+         append
+         (cl-loop
+          for char across input
+          for pos from 0
+          unless (eq char ?-)
+          collect (electric-pair-define-test-form
+                   name
+                   fixture
+                   (aref input pos)
+                   pos
+                   expected-string
+                   expected-point
+                   skip-pair-string
+                   prefix
+                   suffix
+                   extra-desc
+                   mode
+                   bindings
+                   fixture-fn))))))
+
+;;; Basic pairings and skippings
+;;;
+(define-electric-pair-test balanced-situation
+  " (())  " "(((((((" :skip-pair-string "ppppppp"
+  :modes '(ruby-mode))
+
+(define-electric-pair-test too-many-openings
+  " ((()) " "(((((((" :skip-pair-string "ppppppp")
+
+(define-electric-pair-test too-many-closings
+  " (())) " "(((((((" :skip-pair-string "------p")
+
+(define-electric-pair-test too-many-closings-2
+  "()   ) " "---(---" :skip-pair-string "-------")
+
+(define-electric-pair-test balanced-autoskipping
+  " (())  " "---))--" :skip-pair-string "---ss--")
+
+(define-electric-pair-test too-many-openings-autoskipping
+  " ((()) " "----))-" :skip-pair-string "-------")
+
+(define-electric-pair-test too-many-closings-autoskipping
+  " (())) " "---)))-" :skip-pair-string "---sss-")
+
+
+;;; Mixed parens
+;;;
+(define-electric-pair-test mixed-paren-1
+  "  ()]  " "-(-----" :skip-pair-string "-p-----")
+
+(define-electric-pair-test mixed-paren-2
+  "  (])  " "-(-----" :skip-pair-string "-------")
+
+(define-electric-pair-test find-matching-different-paren-type
+  "  ()]  " "-[-----" :skip-pair-string "-------")
+
+(define-electric-pair-test find-matching-different-paren-type-inside-list
+  "( ()]) " "-[-----" :skip-pair-string "-------")
+
+(define-electric-pair-test ignore-different-unmatching-paren-type
+  "( ()]) " "-(-----" :skip-pair-string "-p-----")
+
+(define-electric-pair-test autopair-keep-least-amount-of-mixed-unbalance
+  "( ()]  " "-(-----" :skip-pair-string "-p-----")
+
+(define-electric-pair-test dont-autopair-to-resolve-mixed-unbalance
+  "( ()]  " "-[-----" :skip-pair-string "-------")
+
+(define-electric-pair-test autopair-so-as-not-to-worsen-unbalance-situation
+  "( (])  " "-[-----" :skip-pair-string "-p-----")
+
+(define-electric-pair-test skip-over-partially-balanced
+  " [([])   " "-----)---" :skip-pair-string "-----s---")
+
+(define-electric-pair-test only-skip-over-at-least-partially-balanced-stuff
+  " [([())  " "-----))--" :skip-pair-string "-----s---")
+
+
+;;; Skipping over quotes
+;;;
+(define-electric-pair-test pair-some-quotes-skip-others
+  " \"\"      " "-\"\"-----" :skip-pair-string "-ps------"
+  :test-in-strings nil)
+
+(define-electric-pair-test skip-single-quotes-in-ruby-mode
+  " '' " "--'-" :skip-pair-string "--s-"
+  :modes '(ruby-mode)
+  :test-in-comments nil
+  :test-in-strings nil)
+
+(define-electric-pair-test leave-unbalanced-quotes-alone
+  " \"' " "-\"'-" :skip-pair-string "----"
+  :modes '(ruby-mode)
+  :test-in-strings nil)
+
+(define-electric-pair-test leave-unbalanced-quotes-alone-2
+  " \"\\\"' " "-\"--'-" :skip-pair-string "------"
+  :modes '(ruby-mode)
+  :test-in-strings nil)
+
+(define-electric-pair-test leave-unbalanced-quotes-alone-3
+  " foo\\''" "'------" :skip-pair-string "-------"
+  :modes '(ruby-mode)
+  :test-in-strings nil)
+
+
+;;; Skipping over whitespace
+;;;
+(define-electric-pair-test whitespace-jumping
+  " (    )  " "--))))---" :expected-string " (    )  " :expected-point 8
+  :bindings '((electric-pair-skip-whitespace . t)))
+
+(define-electric-pair-test whitespace-chomping
+  " (    )  " "--)------" :expected-string " ()  " :expected-point 4
+  :bindings '((electric-pair-skip-whitespace . chomp)))
+
+(define-electric-pair-test whitespace-chomping-2
+  " ( \n\t\t\n  )  " "--)------" :expected-string " ()  " :expected-point 4
+  :bindings '((electric-pair-skip-whitespace . chomp)))
+
+
+;;; Pairing arbitrary characters
+;;;
+(define-electric-pair-test angle-brackets-everywhere
+  "<>" "<>" :skip-pair-string "ps"
+  :bindings '((electric-pair-pairs . ((?\< . ?\>)))))
+
+(define-electric-pair-test angle-brackets-everywhere-2
+  "(<>" "-<>" :skip-pair-string "-ps"
+  :bindings '((electric-pair-pairs . ((?\< . ?\>)))))
+
+(defvar electric-pair-test-angle-brackets-table
+  (let ((table (make-syntax-table prog-mode-syntax-table)))
+    (modify-syntax-entry ?\< "(>" table)
+    (modify-syntax-entry ?\> ")<`" table)
+    table))
+
+(define-electric-pair-test angle-brackets-pair
+  "<>" "<" :expected-string "<><>" :expected-point 2
+  :test-in-code nil
+  :bindings `((electric-pair-non-code-syntax-table
+               . ,electric-pair-test-angle-brackets-table)))
+
+(define-electric-pair-test angle-brackets-skip
+  "<>" "->" :expected-string "<>" :expected-point 3
+  :test-in-code nil
+  :bindings `((electric-pair-non-code-syntax-table
+               . ,electric-pair-test-angle-brackets-table)))
+
+(define-electric-pair-test pair-backtick-and-quote-in-comments
+  ";; " "---`" :expected-string ";; `'" :expected-point 5
+  :test-in-comments nil
+  :test-in-strings nil
+  :modes '(emacs-lisp-mode)
+  :bindings '((electric-pair-non-code-pairs . ((?\` . ?\')))))
+
+(define-electric-pair-test skip-backtick-and-quote-in-comments
+  ";; `foo'" "-------'" :expected-string ";; `foo'" :expected-point 9
+  :test-in-comments nil
+  :test-in-strings nil
+  :modes '(emacs-lisp-mode)
+  :bindings '((electric-pair-non-code-pairs . ((?\` . ?\')))))
+
+(define-electric-pair-test pair-backtick-and-quote-in-strings
+  "\"\"" "-`" :expected-string "\"`'\"" :expected-point 3
+  :test-in-comments nil
+  :test-in-strings nil
+  :modes '(emacs-lisp-mode)
+  :bindings '((electric-pair-non-code-pairs . ((?\` . ?\')))))
+
+(define-electric-pair-test skip-backtick-and-quote-in-strings
+  "\"`'\"" "--'" :expected-string "\"`'\"" :expected-point 4
+  :test-in-comments nil
+  :test-in-strings nil
+  :modes '(emacs-lisp-mode)
+  :bindings '((electric-pair-non-code-pairs . ((?\` . ?\')))))
+
+(define-electric-pair-test skip-backtick-and-quote-in-strings-2
+  "  \"`'\"" "----'" :expected-string "  \"`'\"" :expected-point 6
+  :test-in-comments nil
+  :test-in-strings nil
+  :modes '(emacs-lisp-mode)
+  :bindings '((electric-pair-non-code-pairs . ((?\` . ?\')))))
+
+
+;;; `js-mode' has `electric-layout-rules' for '{ and '}
+;;;
+(define-electric-pair-test js-mode-braces
+  "" "{" :expected-string "{}" :expected-point 2
+  :modes '(js-mode)
+  :fixture-fn #'(lambda ()
+                  (electric-pair-mode 1)))
+
+(define-electric-pair-test js-mode-braces-with-layout
+  "" "{" :expected-string "{\n\n}" :expected-point 3
+  :modes '(js-mode)
+  :test-in-comments nil
+  :test-in-strings nil
+  :fixture-fn #'(lambda ()
+                  (electric-layout-mode 1)
+                  (electric-pair-mode 1)))
+
+(define-electric-pair-test js-mode-braces-with-layout-and-indent
+  "" "{" :expected-string "{\n    \n}" :expected-point 7
+  :modes '(js-mode)
+  :test-in-comments nil
+  :test-in-strings nil
+  :fixture-fn #'(lambda ()
+                  (electric-pair-mode 1)
+                  (electric-indent-mode 1)
+                  (electric-layout-mode 1)))
+
+
+;;; Backspacing
+;;; TODO: better tests
+;;;
+(ert-deftest electric-pair-backspace-1 ()
+  (save-electric-modes
+    (with-temp-buffer
+      (insert "()")
+      (goto-char 2)
+      (electric-pair-backward-delete-char 1)
+      (should (equal "" (buffer-string))))))
+
+
+;;; Autowrapping
+;;;
+(define-electric-pair-test autowrapping-1
+  "foo" "(" :expected-string "(foo)" :expected-point 2
+  :fixture-fn #'(lambda ()
+                  (electric-pair-mode 1)
+                  (mark-sexp 1)))
+
+(define-electric-pair-test autowrapping-2
+  "foo" ")" :expected-string "(foo)" :expected-point 6
+  :fixture-fn #'(lambda ()
+                  (electric-pair-mode 1)
+                  (mark-sexp 1)))
+
+(define-electric-pair-test autowrapping-3
+  "foo" ")" :expected-string "(foo)" :expected-point 6
+  :fixture-fn #'(lambda ()
+                  (electric-pair-mode 1)
+                  (goto-char (point-max))
+                  (skip-chars-backward "\"")
+                  (mark-sexp -1)))
+
+(define-electric-pair-test autowrapping-4
+  "foo" "(" :expected-string "(foo)" :expected-point 2
+  :fixture-fn #'(lambda ()
+                  (electric-pair-mode 1)
+                  (goto-char (point-max))
+                  (skip-chars-backward "\"")
+                  (mark-sexp -1)))
+
+(define-electric-pair-test autowrapping-5
+  "foo" "\"" :expected-string "\"foo\"" :expected-point 2
+  :fixture-fn #'(lambda ()
+                  (electric-pair-mode 1)
+                  (mark-sexp 1)))
+
+(define-electric-pair-test autowrapping-6
+  "foo" "\"" :expected-string "\"foo\"" :expected-point 6
+  :fixture-fn #'(lambda ()
+                  (electric-pair-mode 1)
+                  (goto-char (point-max))
+                  (skip-chars-backward "\"")
+                  (mark-sexp -1)))
+
+(provide 'electric-pair-tests)
+;;; electric-pair-tests.el ends here