byte-opt.el addition - optimize list of compile-time constants

[Zack Weinberg]

Consider a construct like the following, which might appear in a major
mode definition.  (This example is from my half-written major mode for
editing GCC machine descriptions.  These are mostly Lisp-like but
contain large blocks of embedded C, which it would be nice to apply
c-mode to.  The MMM library tries to make that possible.  It doesn't
work very well with c-mode right now, but that's irrelevant to this email.)

(when (require 'mmm-auto nil t)
  (mmm-add-classes
   '((md-embedded-c
      :submode c-mode
      :front "^{"
      :back "^}"
      :include-front t
      :include-back t
      ;; If the 'back' } is on a line by itself, include the carriage
      ;; return too; this makes the submode background highlight look
      ;; less strange.
      :back-offset #'(lambda () (when (eq (following-char) ?\n)
                                  (forward-char 1)))
      )))

One would like the anonymous :back-offset function to be
byte-compiled.  This does not happen (with 21.3.1) because the byte
compiler does not look inside a quoted form for lambda expressions.

I can make it get byte-compiled by using ` instead of ':

(when (require 'mmm-auto nil t)
  (mmm-add-classes
   `((md-embedded-c
      :submode c-mode
      :front "^{"
      :back "^}"
      :include-front t
      :include-back t
      ;; If the 'back' } is on a line by itself, include the carriage
      ;; return too; this makes the submode background highlight look
      ;; less strange.
      :back-offset #'(lambda () (when (eq (following-char) ?\n)
                                  (forward-char 1)))
      )))

The byte compiler then sees, after ` gets macroexpanded,

  (list (list
          'md-embedded-c
          :submode 'c-mode
          :front "^{"
          :back "^}"
          :include-front t
          :include-back t
          :back-offset (function (lambda () ...))))

and it *does* look into that for the nested lambda and compile it.
However, the cost of this is a substantially less efficient byte-code
sequence for the outer form.  Instead of having the entire
S-expression in the constant vector, each individual component goes
into the vector, and a rather lengthy byte-code sequence is generated
to execute the (list (list ...)) at runtime.  This does not matter
terribly much for *this* example, which is going to be executed once
when the file is loaded, but I can easily imagine circumstances where
it would matter.  I can work around this by writing (eval-when-compile
...) around the `(...) form, but that's just plain ugly.

I dug through the byte-compiler a bit and determined that it makes no
attempt whatsoever to optimize (list ...) expressions.  So I wrote a
basic byte-optimize-list function, and here it is, suitable for being
slapped into byte-opt.el.

(defun byte-optimize-list (form)
   ; (list) -> nil
  (if (null (cdr form))
      nil

    ; (list X Y Z ...) -> (quote (X Y Z)) when X Y Z are all
    ; compile-time constants.  In addition to the set of things for
    ; which byte-compile-constp is true, detect embedded
    ; (function ...) expressions and compile them, which turns them
    ; into compile-time constants.

    (catch :nonconstant
      (let ((new-form
             (mapcar (lambda (elem)
                       (if (consp elem)
                         (cond 
                                        ; (quote X) -> X
                          ((eq (car elem) 'quote)
                           (car (cdr elem)))
                                        ; (function X) -> byte-compiled X
                          ((eq (car elem) 'function)
                           (byte-compile (car (cdr elem))))
                                        ; ??? (lambda ...) -> byte-compile it
                          ((eq (car elem) 'lambda)
                           (byte-compile elem))
                          (t (throw :nonconstant form)))
                         (if (byte-compile-constp elem)
                             elem
                           (throw :nonconstant form))))
                     (cdr form))))

        ; If we get here, new-form is a list of constants; turn it
        ; into a constant.
        (list 'quote new-form)))))

(put 'list 'byte-optimizer 'byte-optimize-list)

This does have several issues - I'm not proposing it be put in as is.
First, I'm unsure I'm unsure whether it is appropriate to detect and
byte-compile a bare lambda.  The manual dithers on the subject.

Second, I'm not sure why I'm even getting uncompiled lambdas in the
list.  Shouldn't the master byte-opt loop, operating depth-first, have
already done it?  Having to do it here means that if we have a list
which has several lambda expressions followed by something that isn't
a compile-time constant, we'll compile the lambdas, discover the
non-constant, throw away the compiled functions, and have to do them
over again later.

Rather less important is the question of whether I should be
trying to optimize lists that aren't entirely constants.  It is
possible that, e.g. given 

(list 1 2 3 (+ var1 var2) 4 5 6)

it would be appropriate to translate it to

(nconc '(1 2 3) (+ var1 var2) '(4 5 6))

But I can see that being not a good idea, as well, so I didn't do it.

Finally, I'm not sure about coding style.  The catch/let/mapcar/nested
conditionals sequence is rather ugly, IMO, but I couldn't think of a
better way.

Thoughts?

(please cc:, I'm not subscribed to emacs-devel.)

zw