Re: Preview: portable dumper

[Daniel Colascione]

On Feb 16, 2018 7:33 AM, Eli Zaretskii <address@hidden> wrote:

> Cc: address@hidden, address@hidden, address@hidden
> From: Daniel Colascione <address@hidden>
> Date: Fri, 16 Feb 2018 07:10:42 -0800
>
> The original proposal I had in mind was to make a normal data array:
>
> uint8_t dump[15*1024*1024] = { embedded_dump_signature };
>
> And then refer to this array on code. No part of the toolchain is
> allowed to remove this array. The "dump insertion" procedure we're
> considering would just amount to changing the contents of this array in
> an already-linked executable using dumb find-and-replace anchored on
> embedded_dump_signature.

I can understand how rewriting the array could work (but we'd need to
know its size in advance, otherwise we'd waste memory, like Emacs on
w32 does today with its dumped_data array, especially since the size
during bootstrap needs to be so much larger than with byte-compiled
Lisp files). 

I'm not suggesting that the dump array approach is necessarily a good idea. It has other problems --- but memory use isn't one of them. Modern OSes all demand-page their executables. A page isn't loaded until it's actually read. If we have a few megabytes of extra zeros on disk at the end of the array, this disk space waste doesn't translate into wasted memory.

But I'm not sure I understand how references to that
array in the code would work.  Do you mean there would be a need to
"fix up" address references in the code as part of "dump insertion"?
If so, that's my original suggestion from 2 years ago, which was voted
down due to complexities.

Nothing that complex. Right now, pdumper mmaps a dump file into memory. If we instead wrote loaded_dump_start = &dump_array[0], the rest of the pdumper code would keep working just like it does now.

> I like the idea of being able to mv emacs emacs.old, make emacs, and
> then have emacs.old and emacs still both run and work normally. I don't
> think the numbered build mechanism you fixed completely solves the
> problem, since it only works in one specific context.

I imagined the fingerprint we compute is already used (or can be used)
to reject dump files from a different build?  If so, this could
provide at least some protection form calamities.

It does provide that protection. I'm talking about ergonomics, not safety. That the dump-array proposal caps the size of the dump (which will probably break user redumping) and that strip(1) breaks the concatenation approach suggests that maybe the existing separate file option is right after all.