Re: Permission requested to merge branch comment-cache into master.

[Alan Mackenzie]

Hello, John.

On Sat, Mar 12, 2016 at 04:02:28PM -0800, John Wiegley wrote:
> >>>>> Alan Mackenzie <address@hidden> writes:

> > The two approaches are fundamentally different. They aim to do different
> > things, and they do them in very different ways.

> What I don't want to happen is that a disagreement between contributors
> results in the maintenance of two separate solutions to similar problems, just
> because of a difference in design and a lack of agreement.

Tell me, why is this concern being raised only now, after I've invested
a substantial amount of time and energy in fixing the problem?  I was
perfectly open about what I intended to do, the discussion taking place
in bug #22884 (where you were yourself active), and Paul and Eli were
supportive.  Surely that was the time to make representations about the
suitability of some alternative facility?

At this point I have to start asking myself how much I want to do in
Emacs.  If each time I consider tackling a big problem and openly
discuss this with other contributors, I face the prospect of my patches
being rejected for extrinsic reasons, I could perhaps well spend my time
doing other things instead.

> Stefan had the prerogative to include his solution before, so I understand why
> it's in emacs-25 now. But your concerns lead me to believe that his code
> should be revisited. If `syntax-ppss' is not doing the job we need it to do --
> to the extent that you have to implement new code to work around it -- that
> indicate a potential problem.

No, that is a misrepresentation.  I did not "work around" syntax-ppss.
My new code is not in any sense a "work around" - it is a carefully
crafted solution to the problem (see below).  syntax-ppss is simply
unsuitable for use in a primitive, and using it never entered into my
considerations.  It is difficult to understand, difficult to maintain
high level lisp and it calls out to user functions via a hook.  It is
crazy and in bad taste to make a primitive depend on such a thing.  It
is just the Wrong Thing to do.  Anything we use in primitives should be
robust and rigorous, and as far as possible, simple.

The basic problem I've solved is that of comments getting scanned in the
backward direction.  Comments _cannot_, in general, be scanned going
backwards.  The attempt is what gave rise to bug #22884, and quite a few
other bugs before it.  My solution is to scan comments going only
forwards.  This works, is robust, and doesn't require calling out to any
fragile lisp code.  Stefan's idea for a solution is to construe the
problem as something more limited, leave the backward scanning in place,
and just make bug #22884 go away (until the next bug happens).

> I also want to make sure it's not just a documentation failure, or lack of
> communication, before either ditching syntax-ppss, or introducing a new
> mechanism to live alongside it.

> Is it possible to summarize why syntax-ppss is not fulfilling your needs,
> other than it being buggy? Because if that's the only issue, we can fix the
> code.

syntax-ppss does the Wrong Thing.  It purports to return the equivalent
of (parse-partial-sexp (point-min) pos).  Should a buffer be narrowed
such that point-min is inside a string, syntax-ppss's view of the rest
of the buffer gets inverted: strings become non-strings and vice-versa.
For example, in the following syntactically correct C:

    char foo[] = "asdf asdf" "asdf"; /* "asdf" */ /*  */  /*   '"'"  */
                      ^

, narrow it such that point-min is at the marked point.  syntax-ppss
then sees the following strings, but no comments:

    " "
    "; /* "
    " */ /*  */  /*   '"
    "  */      ; no closing string quote

(forward-comment -1) from the end of line, using syntax-ppss, thus
fails.  In comment-cache, of course, it works without trouble.

What I predict is now going to happen is that a different function,
based on syntax-ppss, but returning the equivalent of
(parse-partial-sexp 1 pos) is going to get written.  I also predict that
it will be given the name syntax-ppss, and the 200 or so calls in our
code, and an indeterminate number in users' code are going to be left as
they are to fend for themselves, regardless of the change in
functionality.

How about, as a compromise, merging comment-cache into master for now,
while syntax-ppss gets sorted out, then deciding what to do once that
has happened?

> -- 
> John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
> http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

-- 
Alan Mackenzie (Nuremberg, Germany).