Re: Bug #25608 and the comment-cache branch

[Dmitry Gutov]

Hi Alan,

On 07.02.2017 21:21, Alan Mackenzie wrote:

> > How come the "alternative patch" works well, then?
>
> Well, aside from the fact that it doesn't (IMAO), it is only consulted
> relatively rarely, in certain cases of back_coment where the backward
> scanning hits something it doesn't want to handle.

What is "it"? I would imagine that to be sure that point is not e.g. 
inside a string, the patch would have to consult the cache (or call 
syntax-ppss) at least once per forward-comment call.

From there, I don't really see a real need for backward comment 
scanning. So if you rewrote some code to use forward scanning, that 
approach should be applicable on top of the AP as well.

> > > There's no sign of syntax-ppss being fixed.  Bug #22983 has been open
> > > for almost a year, and despite repeated requests from me, there has been
> > > no movement on it.
>
> > You didn't show any enthusiasm about the initial proposed fix, which was
> > rather simple. Now we've had more discussions, and the bar for a
> > solution has been raised. I'm thinking about it again. Let's not give up.
>
> I wasn't enthusiastic about your proposed fix because I found it ugly.

Thank you for clearing that up.

> That sounds like you've decided you want to use syntax-ppss no matter
> what, and the bugs this will cause will just be relabeled as features.
> As I've said before, the aim should be for back_comment always to work.

More importantly, I want to keep as much logic in Lisp as feasible, 
which is the currently recommended approach anyway.

Problems like this could be solved in different ways without avoiding 
that goal. We can provide new faster primitives if manipulating some 
data structure in Lisp is not enough (but we need benchmarks first, and 
so far, speed is not a problem). We can also add new hooks if 
before-change-functions is not up to snuff.

> > Tracking the used syntax table is also a problem which we need to solve
> > for syntax-ppss. A good design could handle it and narrowing together.
>
> You should now be able to see why I dislike syntax-ppss so much.  As
> well as being incompatible with narrowing (which should be sort of
> fixable), there is an essential lack of cache invalidating (which would
> only be fixable by a radically different design).

Why wouldn't it be fixable with a moderate change in design? The problem 
you are referring to (which is almost entirely theoretical at this 
point, in the absence of bug reports) is cause by syntax-ppss usage 
inside with-silent-modifications.

> > And it does that in a pretty inflexible way.
>
> It works.  Other ways (apart from M-nil (master with
> open-paren-in-column-0-is-defun-start set to nil)) don't.  The sort of
> flexibility I recall you wanting is simply not possible in
> comment-cache,

Why isn't it? It could adhere to narrowing bounds, or not, just as well 
as syntax-ppss. The problems with cache invalidation when narrowing 
changes should be very similar.

> It's differently complicated.  master's back_comment, which attempts to
> scan comments backwards is more complicated than comment-cache's
> back_comment (including its cacheing logic).

Ideally we'd have the best of both worlds, of course. Like mentioned 
above, I see no hard need for backward scanning anymore.

> > Yes, it is worse. You have more code to debug. And comment-cache adds
> > quite a bit of code.
>
> How have you quantified "quite a bit"?

771 insertions(+), 402 deletions? Admittedly, this is not a lot by C 
standards.

> There is nothing to indicate you've even looked at comment-cache.  All

I've looked at it now. Since it's implemented in C, I have little 
ability to judge the quality of the code, or the low-level nuances.

And yet, I've managed to provide coherent comments, haven't I?

> the criticisms you've made have been from a distance, based on rumour
> (even if the source of that rumour has been me).

Discussing design on a high level is a normal practice, and we often do 
so even when the code is available, in the interest of saving time.

> I repeat, you want comment-cache to be
> wholly abandoned, apparently because you like syntax-ppss so much.  The
> alternative "recommended" approach has documented deficiencies, yet you
> still advocate it.

Both approaches have documented deficiencies.

> > So the "speed up forward-comment" patch would still come out to 20 lines.
>
> Well, if you get a decent bug fix involving, say a 700 line patch which
> includes those 20 lines, I suppose you could still call it a 20 line
> patch, somehow.

Even if that takes 700 lines, in the end it will be 700 + 20 lines 
versus 700 + 370 lines that comment-cache takes.

> > > It would also likely be much slower.
>
> > I wouldn't be so sure. A syntax table comparison, for instance, would be
> > pretty cheap compared to what syntax-ppss does already.
>
> Full syntax-table comparisons are slow, even when written in C.

Really? How do you quantify that? In c++-mode,

(benchmark 1000 '(equal (syntax-table) (syntax-table)))

outputs "Elapsed time: 0.004712s". Which is an order of magnitude less 
than (benchmark 1000 '(syntax-ppss)) outputs, in an empty buffer with a 
warmed-up cache.

> I tried
> it back in December.  CC Mode regularly switches syntax-tables.  My
> usual time-scroll function on xdisp.c ran at about half the speed when a
> comparison was done at every set-syntax-table.  The results had to be
> cached, after which it ran at normal speed again.

That doesn't tell me a lot, unfortunately. Maybe it was a design 
problem, e.g. invalidating cache eagerly and too often, instead of doing 
it lazily like syntax-ppss does.

Although CC Mode would have to change syntax tables a lot, for it to 
even show up on the radar.

It's possible that your "compare syntax tables" routine does a lot, of 
course. But if we really need that kind of fuzzy comparison, we can 
implement that function in C and export for using in Lisp.