Re: Preview: portable dumper

[Ken Brown]

On 2/15/2018 9:36 PM, Daniel Colascione wrote:
> On 02/15/2018 05:56 PM, Ken Brown wrote:
> > On 2/15/2018 6:36 PM, Daniel Colascione wrote:
> > > On Feb 15, 2018 3:31 PM, Ken Brown <address@hidden> wrote:
[...]
> > >     I just tried to build on 64-bit Cygwin, and the build fails as 
> > > follows:
> > >
> > >     [...]
> > >     Dumping under the name bootstrap-emacs.pdmp
> > >     dumping fingerprint:
> > >     923050a9f611ad7ead76eea704308e4d05f152601a9134cf8d1b5ff3e0e1a986
> > >     Dump complete
> > >     Byte counts: header=80 hot=13187392 discardable=119424 cold=9086640
> > >     Reloc counts: hot=919268 discardable=5790
> > >     make -C ../lisp compile-first EMACS="../src/bootstrap-emacs.exe"
> > >     make[2]: Entering directory
> > >     '/home/kbrown/src/emacs/x86_64-pdumper/lisp'
> > >         ELC      ../../pdumper/lisp/emacs-lisp/macroexp.elc
> > >         ELC      ../../pdumper/lisp/emacs-lisp/cconv.elc
> > >         ELC      ../../pdumper/lisp/emacs-lisp/byte-opt.elc
> > >         ELC      ../../pdumper/lisp/emacs-lisp/bytecomp.elc
> > >     emacs: could not load dump file "../src/bootstrap-emacs.pdmp": out
> > >     of memory
> > >
> > >     There's probably some obvious explanation, but I don't see it at the
> > >     moment.
> > >
> > >
> > > I'm not entirely surprised to see Cygwin fall over here. We could 
> > > just use the Windows memory mapping functions directly, but I'd 
> > > prefer to stick with the POSIX API if we can make it work.
> >
> > I agree.
> >
> > > Any idea where in the mmap sequence we fail? 
> >
> > I haven't looked at the code yet, so I don't understand the question. 
> > If you give me some guidance, I'll try to investigate.
>
> Thanks. I think the trouble must be in dump_mmap_contiguous. We report 
> all errors from this function as "out of memory". dump_mmap_contiguous 
> takes an array of mapping descriptors (think ELF segments or something) 
> and maps them all into a single contiguous region of virtual memory. On 
> POSIX systems, we reserve a chunk of address space with a big PROT_NONE 
> anonymous mapping, then carve it up into the separate mappings that we 
> really want. Windows doesn't support atomic mmap replacement, so Cygwin 
> has to emulate it, and I wouldn't be surprised if something's going 
> wrong in this emulation.

You're right that the problem is in dump_mmap_contiguous.  I stepped 
through it in gdb and found the following:

1. The call to dump_anonymous_allocate at pdumper.c:4432 succeeds.

2. We enter the loop at pdumper.c:4451 with i = 0.  dump_map_file is 
called, which calls dump_map_file_posix with protection = 
DUMP_MEMORY_ACCESS_READWRITE.

3. This calls mmap at line 4219 with mem_prot = PROT_READ | PROT_WRITE 
and mem_flags = MAP_PRIVATE | MAP_FIXED.

4. This mmap call fails.

I don't know if this is a bug in Cygwin's mmap or simply a limitation 
that has to be worked around.  Should I take this to the Cygwin mailing 
list, or do you have other ideas as to how to proceed?

> That you're able to compile a few elisp files before bootstrap starts 
> failing...

That's an illusion.  I was doing a parallel build, and the output I 
posted simply showed the first few files that emacs was trying to compile.

Ken