bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files'

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> The inner `stream-range' call results in a closure, and I guess that
> this closure includes a reference to the outside VAL, which is the
> stream from one step back (though there isn't a lexical reference to the
> variable...does that make sense?)

This is probably only a part of the puzzle.  Some examples:

test1.el: This is more or less the `stream-range' example reimplemented
without dependencies so that it works in emacs -Q:

#+begin_src emacs-lisp
;; -*- lexical-binding: t -*-

(defun my-range (start)
  (let (forced val)
    (lambda ()
      (unless forced
        (setq val (cons start (my-range (1+ start)))
              forced t))
      val)))

(defun my-test ()
  (let ((range-object (my-range 1))
        current-cons)
    (while (< (car (setq current-cons (funcall range-object))) (* 1000 1000))
      (message "%d" (car current-cons))
      (setq range-object (cdr current-cons)))))

(my-test)
#+end_src

Loading this file test1.el crashes Emacs - but if you compile it,
loading the compiled file doesn't crash.  This is what I expected from
my previous thoughts, because only the uncompiled closures include a
reference to the outer VALs.


test2.el:

#+begin_src emacs-lisp
;; -*- lexical-binding: t -*-

(require 'stream)

(let ((stream (stream-range 1 1000000)))
  (stream-flush stream))
#+end_src

This traverses the whole stream ignoring the elements.  Loading this
file crashes Emacs, no matter if compiled or not.  I'm surprised it
doesn't work even when compiled.


test3.el:

#+begin_src emacs-lisp
;; -*- lexical-binding: t -*-

(require 'stream)

(let ((stream (stream-range 1 1000000)))
  (while (not (stream-empty-p stream))
    (cl-callf stream-rest stream)))
#+end_src

This is semantically exactly like test2.el, only the call to
`stream-flush' has been replaced by literally writing out the
definition.  Nonetheless, the compiled file suddenly doesn't crash Emacs
when loaded.  Loading the uncompiled file test3.el still crashes.

Seems that whether we get a crash or not depends on details in the
implementation of lexical-binding.

Michael.