Re: Preview: portable dumper

[Daniel Colascione]

On 02/16/2018 03:30 AM, Andy Moreton wrote:
> On Fri 16 Feb 2018, Eli Zaretskii wrote:
>
> > > Date: Thu, 15 Feb 2018 15:34:13 -0800
> > > From: Daniel Colascione <address@hidden>
> > > Cc: Eli Zaretskii <address@hidden>, Angelo Graziosi <address@hidden>,
> > >   address@hidden
> > >
> > > I do wonder whether it makes sense to try to copy the dump into the Emacs 
> > > executable itself instead of
> > > leaving it as a separate file. We could do it independently of executable 
> > > format by defining a data array in static
> > > storage that's initially full of, say, 15MB of zeroes prefixed by a long random 
> > > header (like a MIME boundary),
> > > then, after we generate emacs.pdmp, copying the dump file into the executable 
> > > at the place where we see
> > > that random header. If the dump turns out to be bigger than that 15MB, we can 
> > > fail the build and ask the user
> > > to enlarge the array.
> > >
> > > I don't know of any executable format for which this scheme would fail.
> >
> > Wouldn't that make the dumper stuff less portable, in the sense that
> > it would need to be compatible with low-level details of executable
> > file formats on various systems?
> >
> > At least on non-ELF systems, AFAIK the flexibility of putting
> > arbitrary sections into an executable is lower than desired.  For
> > example, before Emacs 25 the MS-Windows build would create a special
> > section for the initialized Emacs data, which had the annoying effect
> > of running afoul of 'strip', because Binutils don't know about this
> > section, and therefore stripping would produce a dysfunctional
> > executable.  It also prevented re-dumping Emacs, something we had in
> > the past and I'd like us to have again in the future.
> >
> > Wouldn't copying the dump into the executable hit the same problems,
> > at least in principle?
>
> I don't understand the desire to put the dump within the eamcs
> executable, as I thought the whole point of this exercise was to avoid
> dodgy manipulation of executable file formats.

It's just a matter of packaging. As I mentioned in the other message I 
sent, keeping the dump separate from the executable breaks some 
perfectly-natural things you might want to do with the Emacs executable, 
like copying and renaming it. If possible, I'd like to keep these things 
working. If we can't, that's a shame, but it's not the end of the world.

We do need to avoid platform-specific executable file manipulation and 
weird runtime hacks. There are three options on the table:

1) Keeping the dump as a separate file,
2) Appending the dump to the end of the executable, and
3) Overwriting a data-section array with the dump contents.

All of these options avoid dodgy platform-specific executable 
modifications of the sort you see in, say, unexw32.c, and they're all 
compatible with emacs as a position-independent executable.

> Anything that fiddles with the executable prevents signing the
> executable, which in many environments is desireable.

Ideally, we'd be able to re-sign after modification.