bug#16526: 24.3.50; scroll-conservatively & c-mode regression

[Alan Mackenzie]

Hi, Martin.

On Mon, Jun 30, 2014 at 09:55:02AM +0200, martin rudalics wrote:
>  >> How can any of these affect the cached start position of a defun before
>  >> the position where the buffer contents were changed?

>  > In this context, the "start of defun" means an outermost paren of some
>  > sort, nothing more, nothing less.  Any of the changes above could disturb
>  > this:  For example, adding an open-paren syntax-table property to a
>  > character which thereby becomes the BOD.  Or temporarily switching to the
>  > Pascal syntax table, where "{" and "}" are comment brackets.  I'm not
>  > saying that something like this will happen on a regular basis, but it's
>  > definitely possible.

> Then you can forget caching.

No.  The particular rogue invocation of scan-lists which caused this
(sub-)thread scans from BOB to near EOB 29 times.  Inside scan-lists,
_NO_ changes to syntax-tables, s-t text properties etc. are possible.
What I'm suggesting is that cacheing that first scan from BOB will be a
big win for many time-consuming scan-listses.

[ .... ]

> I meant that if someone is going to disable `before-change-functions'
> that someone has to take care of flushing the cache.

OK.

>  > An example of what?  What I meant was, each time you used syntax-ppss
>  > from scan-lists, you'd somehow need to be sure that a syntax-table entry
>  > hadn't been changed, etc.  I can't really see how this would be possible.
>  > You'd somehow need to handle constructs like

>  >      (with-syntax-table c++-template-syntax-table
>  >        ....
>  >        (scan-lists (point) -1 1)
>  >        ....)

>  > .

> And how would a defun start cache handle such constructs?

Such a cache would be initialised to nil within a particular invocation
of scan-lists, hence nothing like the above could cause any trouble.

>  > No.  scan-lists works according to its spec even if you start it inside a
>  > comment/string.  It's effect is well defined.  For example if you writen
>  > parens inside a comment, C-M-n and friends do the right thing there.

> Here C-M-n is bound to `forward-list' which according to its doc-string
> "assumes point is not in a string or comment".  `scan-lists' has
> precisely the same problems as `syntax-ppss' wrt syntax changes.

scan-lists is an atomic function, without memory of any state.  It's
functionality is wholly determined by the current buffer state and the
current setting of (quite a few) state variables.

syntax-ppss is motivated by cacheing historical state, hence is
critically different.

>  >> So `syntax-ppss' is at least as primitive as `scan-lists' (especially
>  >> when the latter is used with negative COUNT).

>  > Sorry, but that's utter rubbish.  syntax-ppss is a high-level convenience
>  > function, which if used carefully by the lisp programmer gives correct
>  > results.

> We're not talking about `scan-lists' alone.  We're talking about
> `scan-lists' (or more precisely find_defun_start) with a cache of
> previously calculated positions.

OK, I think I see what you're getting at.  You're thinking of a cache
which would endure over random buffer changes.  The cache I was intending
would be flushed at the beginning of each scan-lists.  "What I tell you
three times is true." (Lewis Carroll, The Hunting of the Snark).

>  > By contrast, scan-lists does precisely what it says, no ifs and no buts.
>  > Even with a negative COUNT.

> Any "problems" of `syntax-ppss' in this regard are the problems of
> scan_lists plus those of maintaining a cache.  No ifs and no buts.

>  >> Anyway, IIUC this implies that CC mode can't work with `syntax-ppss' at
>  >> all.  If that is true, then `open-paren-in-column-0-is-defun-start' was
>  >> indeed the last resort that made editing larger files possible.

>  > CC Mode doesn't use syntax-ppss, but I can't see how that's implied by
>  > anything we've been discussing so far.  It does maintain its own caches
>  > which fulfil the same purpose.  For example, there's a syntactic cache
>  > which treats preprocessor constructs as comments, something syntax-ppss
>  > can't do.

> And how do you invalidate that cache?

It's truncated at buffer changes.  It doesn't contain any information
which could become stale on any of the other sorts of modifications we've
been talking about, or it isn't used when they could hurt anything.  That
this cache has a tightly defined purpose is why it can be robust, whereas
syntax-ppss's cache, being very general purpose, is vulnerable to certain
changes.

>  > open-..-start being t is a kludge which works for certain types of files
>  > and situations and not others.  It was causing hard to fathom errors in
>  > CC Mode, particularly C and C++.

> I can live with such errors.  I can't live with the slowdown.

I'm hoping the slowdown will go away, or at least become tolerable, when
scan-lists is optimised.

>  > Do I take it you're not keen on enhancing find_defun_start in the manner
>  > I suggested?

> What you're asking for is impossible:

> (1) You want to base find_defun_start on scan_lists starting at BOB.

No, on scan_sexps_forward, which (despite the confusing name) is the core
of parse-partial-sexp without its lispy arguments.

>      This means that you need a function that repeatedly calls
>      scan_lists, stopping whenever depth reaches zero and remembers that
>      position, returning the last such position before FROM.  Such a
>      function exists already - it's called `parse-partial-sexp'.

Yes.  Or scan_sexps_forward which is the C core of it.

> (2) You want to avoid that function call scan_lists repeatedly by
>      caching previously found positions.  Such a function exists already
>      - it's called `syntax-ppss'.

I'd have nothing against using syntax-ppss, as long as its cache was
bound to nil for each scan-lists call - except it would cause some
slowdown (compared with a special-purpose cache written in C), since it
goes through the lisp bytecode interpreter.  

> (3) You want that function to work even if someone changes the syntax
>      table or disables `before-change-functions' without flushing its
>      cache.  Such a function does not exist and never will.

No, I want that cache to be flushed for each scan-lists call, so that the
first scan of (nearly) the whole buffer will create the cache, and the
other 28 scans will use that cache.

> martin

-- 
Alan Mackenzie (Nuremberg, Germany).