Re: Refactoring of emacs-lisp/autoload.el

[Alan Mackenzie]

Hi, Stefan!

Thanks for the comments:

On Tue, Aug 12, 2008 at 04:09:04PM -0400, Stefan Monnier wrote:

> > (ii) It produces more consistent result in the comment sections of 
> > loaddefs.el
> >   etc.  In particular:
> >   o - The lines that identify the source file now always (rather than just
> >     sometimes) give a file name relative to the "starting directory" 
> > (usually
> >     .../lisp).  E.g.:

[ .... ]

> I'm not sure why you think it's an improvement: the previous behavior
> was to use a file name relate to where the name was placed (i.e.
> relative to the file in which the entry is created).  In my opinions,
> relative file names placed in files should generally be relative to the
> file in which they appear rather than relative to the "top of the
> project".

It's just I hadn't understood why it was being done.  I see now that it
was being done right, and I've changed my code to do it the same way.

> >   o - The final section (which records files which had no autoload
> >     symbols) no longer includes any files for which there is a normal
> >     section higher up.  For example, in lisp/loaddefs.el at the
> >     moment, "calc/calc-aent.el" violates this rule.  I have assumed
> >     that this is a bug.

> No, this was not a bug.  This section is present to speed up the refresh
> of the loaddefs.el file, so that files that don't have other entries in
> loaddefs.el don't have to be opened&scanned when they haven't changed
> since the last time we refreshed loaddefs.el.
> So calc/calc-aent.el should probably be mentioned in this list so as to
> avoid having to open&scan it when it hasn't changed.

OK.  I haven't yet characterised fully the criterion for a file being in
this list.  I'm looking at that now.

> > (iii) The new autoload.el runs quite a lot faster than the old one.  :-)  
> > Here
> >   are some comparitive timings, done under fair conditions on my 1.2 GHz
> >   Athlon box:

> >   OLD:                                    NEW:
> >   real    1m11.502s                       real    0m40.729s
> >   user    0m55.141s                       user    0m24.519s
> >   sys     0m15.981s                       sys     0m15.998s

> There are 2 ways to run this code: one is to rebuild loaddefs.el from
> scratch, the other is to update a preexisting loaddefs.el (typically
> just after "cvs update").  With "make bootstrap" you care about the
> first, otherwise you care about the second.  I.e. I care about the
> second ;-)

Ah..  I hadn't properly understood the loop through the existing entries
of loaddefs.el.  This is surely to minimise the searching within
loaddefs.el, which is very slow if it's done clumsily.

My strategy when inserting a new entry into loaddefs.el is to start
searching from the current position in that file.  So if the source files
are processed in alphabetical order, the newer autoload.el should be just
as fast (or, at least, fast enough).

> The second case used to be unusably slow and I made it a lot faster
> around Emacs-21 (or so), so now it's fast enough: please test it after
> changing a handful of files, just to make sure that it's not
> significantly slower than before.

I'll test it.  Could you suggest a typical test scenario, please?  How
many source files should I touch to test this?  3, 10, 30, 100?

> >   This is a speedup of ~75%.

> That's great.  In parallel bootstrap builds, the loaddefs.el is
> sometimes on the critical path, so that's a very welcome improvement.

> I can't easily browse your patch with the machine I'm using right now,
> so I can't comment on the actual code.  The current code (mostly due to
> yours truly, sadly) is not very satisfactory indeed, so as long as the
> new code is fully compatible with the various ways autoload.el is used
> (i..e not just by Emacs itself but by other external 3rd party
> packages), and as long as it doesn't significantly slow down the
> "no/few changes" case, I see no reason not to install it,

I agree with all that.  I've been taking as much care as possible.  But I
know nothing of its use by 3rd party packages.  Do you know of any such
uses, or is this a "there are probably some" situation?

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).