Re: [RFC] Replacing malloc_get_state functionality.

[Ondřej Bílka]

On Mon, Feb 24, 2014 at 03:51:47PM -0800, Daniel Colascione wrote:
> On 02/24/2014 12:01 PM, Stefan Monnier wrote:
> >>>This tells 'configure' to assume that malloc_get_state does not work.
> >>Does it still work, though?
> >
> >And doesn't this have some impacts in terms of Emacs's memory use
> >(e.g. retaining more memory instead of returning it via munmap)?
> 
> Based on comments in the code, I'm not sure Emacs can operate
> correctly using the system dlmalloc without malloc_set_state:
> 
> /* malloc can be invoked even before main (e.g. by the dynamic
>    linker), so the dumped malloc state must be restored as early as
>    possible using this special hook.  */
> 
> When DOUG_LEA_MALLOC is unset, we skip some other heap manipulations
> (like restrictions on mmap allocation of lisp objects) that seems
> essential for correctness.
> 
> After the feature freeze, I'd like to look into moving Emacs away
> from sbrk-based allocators in general. It's tremendously wasteful of
> commit charge to keep a large data segment around. (Yes, GNU/Linux
> systems typically overcommit memory, which lessens the impact of the
> waste. Two wrong don't make a right.)
> 
> jemalloc works very well for other long-running programs (like
> Firefox), and although good support for concurrent allocation is one
> of jemalloc's main selling points, its fragmentation avoidance
> features are actually much more attractive to me right now.
> 
> This work will require breaking support on all platforms for dumping
> a dumped Emacs, but I'm not aware of any platform on which this
> feature works anyway.
> 
> Ondřej, are you planning on adding fragmentation resistance to the
> glibc malloc?

Yes, but there is lower hanging fruit around. Most noticable would be
memory savings from omitting headers, currently we add 16byte overhead
per allocation.