Re: When should ralloc.c be used?

[Eli Zaretskii]

> From: Ken Raeburn <address@hidden>
> Date: Wed, 26 Oct 2016 00:36:42 -0400
> Cc: address@hidden
> 
> >>> Using mmap has disadvantages: when you need to enlarge buffer text,
> >>> and that fails (because there are no more free pages/addresses after
> >>> the already allocated region), we need to copy buffer text to the new
> >>> allocation.  This happens quite a lot when we visit a compressed
> >>> buffer.  (The MS-Windows emulation of mmap in w32heap.c reserves twice
> >>> the number of pages as originally requested, for that very reason.)
> >> 
> >> In the general case, yes.  But modern Linux kernels have an “mremap” 
> >> system 
> >> call which can “move” a range of pages to a portion of the address space 
> >> that 
> >> can accommodate a larger size, by tweaking page tables rather than copying 
> >> all 
> >> the bits around.  I’m pretty sure modern glibc realloc uses it.
> > 
> > AFAIU, this feature will only help us if someone adds code to use it
> > in buffer.c:mmap_enlarge.  Or are you saying that the OS will call
> > mremap for us automatically when mmap_enlarge attempts to map
> > additional pages at the end of an mmaped region?
> 
> It could be done explicitly, but my experience was that malloc/realloc would 
> just do it for us; we’d just have to use malloc/realloc instead of explicitly 
> calling mmap.

I think we've lost context of the discussion.  Please see above: this
is about the disadvantages of using mmap directly, i.e. for those
cases where the native malloc or gmalloc suffer from memory
fragmentation, and we decide to use mmap in buffer.c to countermand
that.

I've pointed out the disadvantages of using mmap directly, and you
mentioned the mremap syscall as the counter-argument.  If you thought
I was talking about problems mmap could cause to the malloc
implementation, then that's a misunderstanding: I was explicitly
talking about using mmap directly for allocating buffer text.  My
point was that we should only use mmap if necessary, as it comes for a
price.

> A man page browser at freebsd.org for several platforms seems to indicate 
> that 
> NetBSD has picked it up, but neither FreeBSD nor OpenBSD.  I don’t know if 
> NetBSD’s realloc will use it, but it’s certainly simpler if we just ignore 
> mremap for explicit use, and just bear in mind that realloc may not always 
> have 
> to pay the expected copying penalty on all systems….

Once again, this is about the cases where using malloc for buffer text
gives unsatisfactory results, and mmap is being considered as a
remedy.