Re: Preview: portable dumper

[Ken Raeburn]

> On Dec 1, 2016, at 13:11, Eli Zaretskii <address@hidden> wrote:
> 
>> From: Richard Stallman <address@hidden>
>> CC: address@hidden, address@hidden, address@hidden,
>>      address@hidden
>> Date: Thu, 01 Dec 2016 04:18:04 -0500
>> 
>> Rather than arguing theoretically about speculations,
>> let's see people do their best to improve the big-elc-file
>> approach, and then compare facts with facts.
> 
> Agreed.
> 
> Ken, will you be able to continue working on your branch in this
> direction?  Are all the changes suggested/tried by Stefan already on
> that branch?  If not, would it be possible for Stefan or yourself to
> update the branch?

I have an update I’m still working on, to refine the hash table handling I did 
to generate less garbage while reading normal .el or .elc files.  I hope to get 
that uploaded soon.  And I think my previous push of the branch was before 
Stefan’s latest patch (Oct 31?), so I’ll pull that in too, unless he’s done 
some more work since.

My focus has been on the dumped.elc load performance; the content generation 
has been pretty much all his work.

Stefan’s work on reducing “#n#” placeholder substitutions tackled the 
particular case of cons cells, and he suggested extending it to other types, 
but based on some stats I got looking over the big .elc file, I think cons 
cells and strings are the majority of the cases.  Stefan’s code addresses the 
cons cells, and for property-less strings (and certain other types) the 
substitution pass isn’t needed, so I think we’ve addressed the biggest gains to 
be had in that area.

The patch I made to use a pair of hash tables for tracking previously read 
objects seems to have improved performance as well, but I’ve started to refine 
it a bit to not create a new pair of hash tables for every top-level read done, 
when the majority of the calls (over 95% during a loadup.el pass I 
instrumented) don’t actually wind up needing the hash tables.

Regardless of the outcome of this current disagreement and whether big-elc gets 
used, I think the lread changes may soon be ready to merge, even if the 
performance benefits are less drastic for normal Lisp code.

As for non-performance issues, last time I tried the big-elc code, it didn’t 
work for me in batch mode because of a null pointer dereference in face 
processing; I haven’t had a chance to look into it further but maybe in the 
next few days I can.

Ken