[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Preview: portable dumper
From: |
Daniel Colascione |
Subject: |
Re: Preview: portable dumper |
Date: |
Mon, 28 Nov 2016 15:17:38 -0800 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/26.0.50 (gnu/linux) |
On Mon, Nov 28 2016, Stefan Monnier wrote:
>> I've been working on a portable dumper for GNU Emacs. The attached patch is
>> an early version of this work.
>
> Cool!
>
>> An additional optimization follows, although this part isn't implemented
>> yet: we can define a "preferred load address" for the dump and write
>> relocation information such that if the dump and Emacs end up being where we
>> expect them to be, we don't have to perform any relocations at all.
>
> Oh, so the in-file format is also a valid in-memory format? Great.
> AFAIK, that's the crucial difference between a "fast .elc loader" and
> your approach. This should be key to get really good performance
> (e.g. good enough that it doesn't slow down bootstrap noticeably).
Yes. The file is full of valid memory images of Lisp objects --- that's
why I'm guessing it might be a bit more verbose than ELC,
pre-compression. If you guess correctly at dump time what addresses the
Emacs and dump images will have, you can direct mmap it directly and
run. With a non-PIC executable, you can do a pretty good job of that
and get performance a bit better than unexec, in theory.
If you load a dump and find that you've guessed wrong on the base
addresses, you need to relocate the image, but that can be done cheaply
and gradually, on access.
>> would benefit from compression, which will reduce IO loads: LZ4 reduces the
>> dump size for me from ~12MB to ~4MB. As in the mmap case, we can throw away
>
> Hmm... interesting the "dump one big .elc" approach I played with (which
> kinda works right now, tho with the known performance issues) leads to
> a dumped.elc file of a size a bit below 4MB (which gzip compresses down
> to ~1MB).
>
> I assume your system is using 64bit Lisp_Object, right? So on a 32bit
> Lisp_Object your dump size is probably closer to 6-7MB.
Right --- I've tested on 64-bit only. I imagine testing 32-bit and
wide-int might reveal some latent bugs.
>
>> functions pdumper.h --- but even so, it's within 100ms or so of an
>> unexeced Emacs.
>
> 0.1s will still lead to a noticeable slow down of bootstrap, but at
> least it should be good enough for the usual end-user who doesn't run
> Emacs in batch mode.
Oh, of course. But that's without any attention paid to
optimization. I'm sure it can go much faster.
>> It's also possible to dump an already-dumped Emacs, so it should be possible
>> for users to have their own dump files.
>
> [ FWIW in theory my dumped.elc approach could also work on an
> already-dumped Emacs, for use by end-users (it basically just prints the
> contents of the obarray), tho it would indeed need extra work to weed out
> problems printing objects like markers and such. ]
Yeah --- then both approaches get into the problem of how to dump
arbitrary object graphs, which in practice ends up being "dump the whole
heap except for processes, windows, and frames."
- Re: Preview: portable dumper, (continued)
- Re: Preview: portable dumper, John Wiegley, 2016/11/30
- Re: Preview: portable dumper, Daniel Colascione, 2016/11/30
- Re: Preview: portable dumper, John Wiegley, 2016/11/30
- Re: Preview: portable dumper, Daniel Colascione, 2016/11/30
- Re: Preview: portable dumper, John Wiegley, 2016/11/30
- Re: Preview: portable dumper, Philippe Vaucher, 2016/11/30
- Re: Preview: portable dumper, Daniel Colascione, 2016/11/30
- Re: Preview: portable dumper, Paul Eggert, 2016/11/30
Re: Preview: portable dumper, Paul Eggert, 2016/11/28
Re: Preview: portable dumper, Stefan Monnier, 2016/11/28
- Re: Preview: portable dumper,
Daniel Colascione <=
- Re: Preview: portable dumper, Stefan Monnier, 2016/11/29
- Re: Preview: portable dumper, Daniel Colascione, 2016/11/29
- Re: Preview: portable dumper, Paul Eggert, 2016/11/29
- Re: Preview: portable dumper, Daniel Colascione, 2016/11/29
- Re: Preview: portable dumper, Paul Eggert, 2016/11/29
- Re: Preview: portable dumper, Daniel Colascione, 2016/11/29
- Re: Preview: portable dumper, Paul Eggert, 2016/11/30
- Re: Preview: portable dumper, Stefan Monnier, 2016/11/30
Re: Preview: portable dumper, Richard Stallman, 2016/11/30
Re: Preview: portable dumper, Daniel Colascione, 2016/11/30