bug#16184: 24.3.50; edebug and eval-when-compiler don't work together

[Alan Mackenzie]

Hello, Anders.

On Wed, Dec 18, 2013 at 11:58:23AM +0100, Anders Lindgren wrote:
> Hi!

> The construct 'eval-when-compile' seems to throw debug off. When pressing
> C-u C-M-x on the following, an error is issued. This worked correctly in
> 24.3. This makes it hard to use edebug on functions in cc-mode
> (like c-font-lock-declarations), as it makes heavy use of eval-when-compile.

> (defun test (limit)
>   (eval-when-compile
>     (boundp 'parse-sexp-lookup-properties)))

> Sincerely,
>     Anders Lidgren

[ .... ].

This has been bugging me off and on for years, too.  If I'd noticed your
bug report back in 2013, I might have got round to looking at it then,
but I thought it was just me that was suffering.  :-(

I think I now understand what's happening, and I have a tentative fix,
though this might be more of a workaround than fix, I'm not sure.

Anyhow, this is the form I've been testing with:

    (defun test (limit)
      (let ((foo
             (cc-eval-when-compile
               (boundp 'parse-sexp-lookup-properties))))))

, which clearly needs (require 'cc-defs) before it if you don't have CC
Mode loaded already.

#########################################################################

Essential background information
================================

When edebug instruments a function, it inserts a call to edebug-enter at
the start of the function.  Regularly throughout the function, it inserts
calls to edebug-before and edebug-after, in forms which look something
like:

    (edebug-before 2)
    (edebug-after (edebug-before 2) 3 ...)

.  edebug maintains a stack of numerical "offsets" (whatever they are)
called edebug-offset-indices, consing a 0 onto it in edebug-enter, each
time an instrumented function is entered.  At each edebug-before or
edebug-after, the top element of this stack is modified with:

    (setcar edebug-offset-indices before-index) ; where before-index is,
say 2.

The offset at the top of edebug-offset-indices is used to position point
each time edebug displays the buffer being debugged before waiting for
user input.

#########################################################################

The bug mechanism is this: The macro-expander working on
cc-eval-when-compile, and edebug's instrumentation produce a form
something like this:

    (edebug-enter 'test (limit)
                  (function (lambda ......
                   (cc-eval-when-compile
                    (edebug-after (edebug-before 2) 3
                                  (boundp 'parse-sexp-lookup-properties)
                   .....)))))

.  During this instrumentation, the inside of the cc-eval-when-compile
form gets run, in particular, the (edebug-before 2).  At this stage,
since edebug-enter hasn't been run at all, edebug-offset-indices is still
nil.  The (edebug-before 2) thus attempts (setcar nil 2), which throws
the error.

#########################################################################

My proposed solution/workaround is to initialise edebug-offset-indices to
'(0) rather than nil, giving edebug-before/after a cons to work on, even
when they're called "at the top level".

This seems to work, sort of.  In my test form above, when I do C-u C-M-x,
the function `test' gets successfully instrumented.  However, if I
replace cc-eval-when-compile with eval-when-compile and do C-u C-M-x, the
behaviour is different: during the instrumentation, edebug steps through
the `boundp' form.  I don't yet understand where this difference comes
from.

So, here's the patch, which I'd be grateful if you would try out.  It's
based on the savannah master:



diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
index 66117b8..5dc6f89 100644
--- a/lisp/emacs-lisp/edebug.el
+++ b/lisp/emacs-lisp/edebug.el
@@ -2052,9 +2052,9 @@ edebug-stack
 (defvar edebug-stack-depth -1)
 ;; Index of last edebug-stack item.
 
-(defvar edebug-offset-indices nil)
+(defvar edebug-offset-indices (list 0))
 ;; Stack of offset indices of visited edebug sexps.
-;; Should be nil at the top level.
+;; Should be nil at the top level.  No longer!  Extensive comment needed here.
 ;; Each function adds one cons.  Top is modified with setcar.
 
 
This patch clearly needs commenting added to it, but I'm in much too
prolix a frame of mind to attempt that now.  ;-(

-- 
Alan Mackenzie (Nuremberg, Germany).