bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files'

[Noam Postavsky]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> #+begin_src emacs-lisp
> ;; -*- lexical-binding: t -*-
>
> (require 'stream)
>
> (let ((stream (stream-range 1 1000000)))
>   (while (not (stream-empty-p stream))
>     (cl-callf stream-rest stream)))
> #+end_src
>
> This is semantically exactly like test2.el, only the call to
> `stream-flush' has been replaced by literally writing out the
> definition.  Nonetheless, the compiled file suddenly doesn't crash Emacs
> when loaded.  Loading the uncompiled file test3.el still crashes.

Aha, but the following also crashes, whether compiled or not:

;; -*- lexical-binding: t -*-

(require 'stream)

(let* ((stream0 (stream-range 1 1000000))
       (stream stream0))
  (while (not (stream-empty-p stream))
    (cl-callf stream-rest stream)))

So the problem is that the initial stream0 object can reach the entire
unfolding stream as it goes, and just holding on to that reference is
enough to keep the whole thing in memory.

Now, I can see that letting stream0 automagically get access to the
unfolded result can be an optimization in some cases, although in this
case it's a pessimization.  It could also affect the semantics if
unfolding the stream has side effects, not sure if stream.el makes
guarantees about that though.