[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files'
From: |
Michael Heerdegen |
Subject: |
bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' |
Date: |
Wed, 28 Feb 2018 11:58:49 +0100 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) |
Noam Postavsky <npostavs@gmail.com> writes:
> >> #+begin_src emacs-lisp
> >> (seq-doseq (_ (stream-range 1 1000000)) nil)
> >> #+end_src
> >>
> >> Note that this is executed as a loop due how to streams are
> >> implemented, although the definition of `seq-doseq' looks
> >> recursive.
>
> Doesn't look recursive to me, it expands to a call to seq-do, which uses
> a simple loop.
I was imprecise, I meant that the streams are defined recursively (most
of the time). Though, it's a delayed type of recursion. Anyway, I
think that this doesn't matter here.
> > Does anyone has a reasonable idea for avoiding the crash in such
> > programs?
>
> I don't have a quick answer for the general case, but I think it's a bug
> in stream.el that it's creating such large structures in the first
> place. As far as I understand it, the point of streams is to handle
> long lists by encoding them as
>
> (FIRST-VALUE . FUNCTION-TO-PRODUCE-REST-OF-LIST)
Yes, that's exactly how it's implemented. When requesting more elements
from the stream, that becomes
(FIRST-VALUE .
(SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST))
When we loop over the string, the cons whose car is the FIRST-VALUE,
let's call it cons1, is immediately thrown away, and we continue with
(SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST)
etc.
AFAIU the problem is that the cons1 still exists in memory until garbage
collection kicks in. When that happens, the cons1 points to a largely
recursive cons structure, though this structure is never referenced from
Lisp in this form.
> so as to avoid allocating large amounts of memory. Is there an easy way
> to find out what the large structures are, and where they are coming
> from?
I think I've answered that. At least, I think so. What I don't
understand is that when I force the `seq-doseq' to call
`garbace-collect' explicitly every 1000 cycles, or so, it doesn't help:
the crash still happens after generating ~ 70 000 elements, or some
more, but I can't avoid the crash, no matter how often I call gc. So
I'm not sure whether these long lists are the problem or something else.
The FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST looks harmless when I print
it, even after thousands of iterations, so I would not understand why
that could be problematic. streams.el implements things in a way that
these rest functions are not deeply nested lambdas.
Michael.
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files', Michael Heerdegen, 2018/02/27
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files', Eli Zaretskii, 2018/02/27
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files', Michael Heerdegen, 2018/02/27
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files', Michael Heerdegen, 2018/02/27
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files', Eli Zaretskii, 2018/02/27
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files', Noam Postavsky, 2018/02/27
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files',
Michael Heerdegen <=
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files', Eli Zaretskii, 2018/02/28
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files', Michael Heerdegen, 2018/02/28
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files', Eli Zaretskii, 2018/02/28
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files', Michael Heerdegen, 2018/02/28
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files', Michael Heerdegen, 2018/02/28
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files', Nicolas Petton, 2018/02/28
- bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files', Eli Zaretskii, 2018/02/27