Debugging emacs memory management

[Dima Kogan]

Hi.

I'm running emacs snapshots from the latest git, usually as a
long-running daemon. For a few months now I've had to restart my emacs
session every week or so because it eats all my RAM, and cycling emacs
is the only way to get the RAM back. So two main questions:

1. How does one debug this? There's a memory profiler (launched with
(profiler-start) ), but it only instruments allocations, which is
useless for hunting leaks. Past that, there's some mailing list wisdom
such as this:

 https://lists.gnu.org/archive/html/emacs-devel/2008-12/msg00782.html

It turned up some variables that it claimed were about 20MB long, not x
GB, but admittedly it's not obvious to me that 

 (length (prin1-to-string (symbol-value sym)))

is an accurate measure of the memory footprint of a variable, especially
if it represents something like a hash table.

That's about it, though. Are there better techniques available now? Are
there obvious ways to improve the profiler? Should I look at it?



2. This is specifically about my problem, so maybe this is better as a
bug report; putting it here anyway.

As a test, I tried to split out my ERC and mu4e use to a different
session and that made no difference, so it's not those. As a random
guess, I have a sequence that appears to leak, but I'm not 100% sure how
this is supposed to work. As I ran through this sequence, I looked at
resident memory with this:

 while (true) { sleep 1; ps -o rss `pidof emacs-snapshot` | grep 
--line-buffered -v RSS }

init.el only has this line:

 (global-set-key (kbd "C-S-w") 'delete-region)

Sequence:

1.  seq 1000000 > /tmp/dat                  [ create large file on disk ]
2.  emacs-snapshot                          [ 32MB RSS ]

3.  open /tmp/dat                           [ 40MB ]
4.  (delete-region (point-min) (point-max)) [ 40MB ]
5.  (undo)                                  [ 47MB ]
6.  (kill-buffer)                           [ 40MB ]
7.  (garbage-collect)                       [ 40MB ]

3a. open /tmp/dat                           [ 46MB ]
4a. (delete-region (point-min) (point-max)) [ 60MB ]
5a. (undo)                                  [ 60MB ]
6a. (kill-buffer)                           [ 60MB ]
7a. (garbage-collect)                       [ 60MB ]

3b. open /tmp/dat                           [ 60MB ]
4b. (delete-region (point-min) (point-max)) [ 60MB ]
5b. (undo)                                  [ 60MB ]
6b. (kill-buffer)                           [ 60MB ]
7b. (garbage-collect)                       [ 60MB ]

So here it looks like 28MB vanished. Is this a leak? I'm using
(delete-region) specifically because it doesn't move anything to the
kill ring. The undo information shouldn't persist either because the
buffer is being killed.

Without knowing the internals, it looks like a leak, but I can't be
sure. If I move around the /tmp/dat buffer, and (delete-region) various
other chunks, I can eat up even more memory. The exact usage values
aren't consistent every time, but it always looks like it leaks. Also,
'emacs -Q' shows the same behavior, just not as badly. Advice?

dima