Re: Idea for syntax-ppss. Is it new? Could it be any good?

[Alan Mackenzie]

Hi, Stefan!

On Sat, Jul 26, 2008 at 09:34:45PM -0400, Stefan Monnier wrote:
> > How about reimplementing it thusly?:  The current syntax would be cached
> > for positions at every N bytes (where N would be, perhaps 1024, possibly
> > 8192).  A call to syntax-ppss would simply call parse-partial-sexp from
> > the latest valid cached position, filling out the cache as it goes.  Any
> > buffer change would invalidate cached values for N > POS.

> Isn't that what syntax-ppss does?

It caches the state for several positions, but I don't think they're at
regular positions.  I don't understand the detailed workings of the
routine at the moment.  I suspect that the slowness of all the lisp
manipulation will outweigh the benefit of the caching, but I would
confirm or refute that with the profiler before doing anything serious.

partial-parse-sexp is blindingly fast.  To scan an entire 3Mb C buffer
on my elderly 1.2 GHz Athlon takes 0.27s.  That is why I suspect that
the lisping in syntax-ppss might need severe optimisation.  But again,
it's only a hunch.

What I think really needs doing is to make this function bulletproof: It
should work on narrowed buffers, it should give reliable elements 2 and
6, its cache should be cleared when functions like `modify-syntax-entry'
are called or parse-sexp-lookup-properties is changed, and the cache
should be bound to nil on `with-syntax-table'.  I actually think it
could be useful to maintain several parallel caches, each for a
different syntax-table (or an equivalence class of syntax tables).  And
so on.  Basically, I would like `(syntax-ppss)' to tell me with 100%
reliability, no ifs, no buts, whether I am at top-level, in a comment,
or in a string.

Also, Lennart is asking for it to work nicely with multiple major modes.
Surely this would be a Good Thing.  Files containing several major modes
are commonplace (awk or sed embedded within a shell script, html
embedded within php, ....).

At the moment, CC Mode applies a heuristic maximum size of strings and
comments, for performance reasons.  Checking for strings and comments is
done so frequently that the mode uses elaborate internal caches.  It
would be nice if this cacheing could move to the Emacs core.

Again, this isn't something which can be implemented in a weekend, but I
think it would be worthwhile for Emacs 24.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).