Re: Debugging memory leaks/stale references

[Florian Weimer]

* Richard Stallman:

> It might be interesting to keep track of malloc'ing of large blocks of
> memory, recording the last N large blocks allocated in a table.
> Or perhaps only those that do not hold conses, miscs, small strings, etc.
> This way you might be able to find some large blocks and then
> examine them to see what data is in them.

The approach which I have used with great success in the past is to
use GNU libc's malloc() hooks to print complete
malloc()/realloc()/free() traces.  The crucial point is to include
information from backtrace() (because otherwise, we only see calls to
xmalloc()).  A short Python (or whatever) script is then used to
discover blocks which are never freed, sums up the leakage by
backtrace, and uses addr2line to print a human-readable backtrace.

There is a tool, called valgrind, that uses dynamic recompilation of
x86 code to inject such tracing code without recompilation and
relinking.  It can detect memory leaks, but it catches a wide range of
programming mistakes, too.  Unfortunately, a dumped Emacs does not run
under valgrind, so I could use it.  Therefore, I'm going to post my
enhanced alloc tracing patch in a separate message for future
reference.

Now to the interesting part.  The worst offender is
decode-coding-region:

* 42212224
  dump_hook_malloc /home/fw/src/gnu/emacs-memory-dump/src/alloc.c:5740 
  ?? ??:0 
  coding_allocate_composition_data 
/home/fw/src/gnu/emacs-memory-dump/src/coding.c:1699 
  code_convert_region /home/fw/src/gnu/emacs-memory-dump/src/coding.c:5650 
  code_convert_region1 /home/fw/src/gnu/emacs-memory-dump/src/coding.c:6966 
  Fdecode_coding_region /home/fw/src/gnu/emacs-memory-dump/src/coding.c:6983 
  Ffuncall /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2743 
  Fbyte_code /home/fw/src/gnu/emacs-memory-dump/src/bytecode.c:686 
  funcall_lambda /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2928 
  Ffuncall /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2803 
  Fbyte_code /home/fw/src/gnu/emacs-memory-dump/src/bytecode.c:686 
  Feval /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2097 
  Fcondition_case /home/fw/src/gnu/emacs-memory-dump/src/eval.c:1293 
  Fbyte_code /home/fw/src/gnu/emacs-memory-dump/src/bytecode.c:864 
  Feval /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2097 
  Fcondition_case /home/fw/src/gnu/emacs-memory-dump/src/eval.c:1293 
  Fbyte_code /home/fw/src/gnu/emacs-memory-dump/src/bytecode.c:864 
  funcall_lambda /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2928 
  Ffuncall /home/fw/src/gnu/emacs-memory-dump/src/eval.c:2803 

It turns out that we are returning from code_convert_region() without
freeing the composition data under certain circumstances.  This bug is
not new, it has been present since code_convert_region() started
allocating data using xmalloc() in late 1999.  The recent
internationalization efforts just uncovered it.

The patch below is for illustration only.  It fixes the memory leak,
but maybe other cleanups are missing on this path.

2004-09-28  Florian Weimer  <address@hidden>

        * coding.c (code_convert_region): Free composition data to fix
        memory leak.

--- emacs-upstream/src/coding.c 2004-09-27 21:32:39.000000000 +0200
+++ emacs-memory-dump/src/coding.c      2004-09-28 22:26:13.000000000 +0200
@@ -5667,6 +5667,7 @@
          if (!replace)
            /* We must record and adjust for this new text now.  */
            adjust_after_insert (from, from_byte_orig, to, to_byte_orig, len);
+         coding_free_composition_data (coding);
          return 0;
        }
 

(Disclaimer: I haven't got papers on file, but there is just one way
to make this change, so I doubt it is copyrightable.)