Re: Preview: portable dumper

[Daniel Colascione]

On Mon, Nov 28 2016, Eli Zaretskii wrote:
>> From: John Wiegley <address@hidden>
>> Cc: Paul Eggert <address@hidden>,  address@hidden,  address@hidden
>> Date: Mon, 28 Nov 2016 12:47:10 -0800
>> 
>> >>>>> "EZ" == Eli Zaretskii <address@hidden> writes:
>> 
>> EZ> I'm not going to argue. If enough people think I'm mistaken and want to 
>> go
>> EZ> the portable dumper way, I will resign right here and now. It is very 
>> easy
>> EZ> to convince me to step down, because I hesitated to take this job to 
>> begin
>> EZ> with.
>> 
>> EZ> Your call, crowd.
>> 
>> OK, please let's take a few steps back and not make any rash decisions
>> today. :)
>
> It's no longer my decision.
>
>> If Daniel's proposing an internal optimization and is willing to maintain it
>> -- and if it doesn't make work for other contributors -- I suggest we accept
>> it as long as he, or someone else, is willing to support it.
>
> Emacs future cannot depend on a single person.

We are a long way away from the day when only one person left on earth
can simultaneously hold in his mind the concepts of addition, modulus,
and binary search in C. The operating principle of this work is very
simple. If we need internals documentation, I'll write documentation ---
but there's really not much to it.

> We had more than once
> a situation where key developer(s) left leaving a feature or a whole
> area unmaintained.  If some development isn't a one-time job -- and
> this one surely isn't -- then experience shows that any arcane code in
> C that loses its experts will become unmaintained and unmaintainable.
> The only hope for us to escape this fate is to do it in a way that
> most contemporary contributors understand and can easily learn to
> hack, which means do it mostly in Lisp, reusing the existing C
> machinery that is proven by time and doesn't need maintenance --
> 'load' and its cousins.

They don't need maintenance --- except that they're at least an order of
magnitude too slow right now, and nobody's stepped up to do the
maintenance work necessary to make them useful in a new use case.

> That was what I proposed.  The advantage of
> that is that Emacs then becomes a normal program, one that doesn't
> dump itself in any way, shape, or form, but instead reads in a
> byte-compiled file, using the normal memory allocation routines.  No
> more need to save any internal state, worry about relocations, ASLR,
> etc.  It amazes me that the advantages of that are not immediately
> clear to everyone, but I guess I'm biased.

The whole point of the portable dumper is that you don't need to worry
about ASLR, ASAN, TSAN, or whatever libc people dream up next. Emacs
*is* a normal program --- it does only the things normal programs are
allowed to do and that platforms support, things like pointer assignment
and reading from a file. A "relocation" is literally just addition ---
it's not some horrible dynamic programming monstrosity like redisplay,
which I understand used to literally have a skull and crossbones in
ASCII art over the main body of the code.

Using your arguments, we should switch to Pango for text layout, FUSE
for remote access, clang-format for indentation, and --- hell, let's
just make Emacs a webapp. If you want maximum technological
democratization, you want JavaScript. See https://atom.io/.

>> At the very least, it frees us from dependence on a glibc feature
>> that is doomed to disappear.
>
> No one is suggesting to stay with unexec.  The disagreement is about
> where to go to get rid of it.
>
>> If later the work becomes difficult to support, or if Daniel disappears, 
>> won't
>> we just be returning to the same position as now? If that happeans, we can
>> take what we've learned and try another approach, such as speeding up .elc
>> loading. Or, if someone comes up with a better alternative in the meantime, 
>> we
>> can just switch to it.
>
> I don't think this is practical.  We cannot go back, certainly not
> when unexec requires the expertise it does.  No one will agree to go
> back.  Like with unexec, the only way the portable dumper will be
> thrown away is if/when it turns out it is an obstacle to Emacs
> development, by which time it will be too late.

Unexec will keep working as well as it does today.


>> Accepting this patch doesn't mean we don't try the .elc speedup idea. The 
>> only
>> thing we have to make sure is that it doesn't unduly increase the difficulty
>> of adding new Lisp objects.
>
> Accepting this patch means we believe in this direction, consider it a
> perspective one, enough so that we are willing to invest a non-trivial
> amount of energy into making it happen, fix the fallout, debug the
> results, etc.  We are talking about at least one major release, maybe
> more, and definitely several minor releases.  IOW, we are talking
> several years.  I don't think we should gamble with such high stakes,
> not unless we sure we want to go there, because this is the best
> alternative of ditching unexec.

The stakes aren't high. As johnw wrote, unexec will keep working as well
as it does today, and nobody is stopping anyone else making