RE: Clarification needed: syntax-propertize vs font-lock-syntax-table

[Wedler, Christoph]

> Note that calling (syntax-propertize (point-max)), as js-mode was doing, 
> is not a guarantee of the file staying entirely up-to-date WRT syntax 
> properties as the user edits it.

I now checked the code: it is done via jit-lock-after-change in
after-change-functions.  As I have guessed, syntax-propertize currently
only works correctly if font-lock is switched on.

> Neither is using font-lock: we fontify the file lazily, and thus also 
> only propertize the buffer near its visible part.

Yes, but navigation command like M-x forward-list do not depend on
font-lock, but they do depend on syntax-propertize (consider parentheses
in generic strings).

> But nothing has changed in Emacs 25 in that regard (I think?). We 
> changed js-mode behavior, but mostly because it was a kludge anyway, and 
> not because anything else is compensating for it now.

Well, then I think that the change in js is wrong.  (Yes, a fully lazy
solution is better, but as long as it is not there...)

>> The other documentation request was for font-lock syntax-table as being
>> "syntax-ppss-compatible" (as you put it nicely).

> I'd be happy to review the documentation patch. We can also ask Stefan 
> to do that.

OK, I send a suggestion later this week, together with:

>> Agreed.  How about a function like
>>
>> (defun syntax-ppss-open-list-positions (ppss)
>>    "Get all syntactically open list position found in a syntactic scan.
>> PPSS is a scan state, as returned by `parse-partial-sexp' or `syntax-ppss'.
>> The start position of the outermost list comes first."
>>    (nth 9 ppss))
>>
>> This function needs to be changed if the internal structure of (nth 9
>> ppss) is changed.

> Yes, that sounds good (maybe choose a shorter name?). But see above.

Ok, I'll name it syntax-ppss-open-positions (like (nth 0 ppss) is called
syntax-ppss-depth, and not syntax-ppss-list-depth).

>> We might need something (an extra syntax flag?) which makes it easy to
>> "rewind" the ppss such that
>>
>>    (equal (syntax-ppss inner-mode-end-position)
>>           (syntax-ppss inner-mode-start-position))

> That sounds like it might require turning the current "list of openers" 
> structure into a list-of-lists, where each element is the current "list 
> of openers", plus some boundary information.

> Or maybe handle that in some entirely different way, like keeping the 
> current spss format, but allow the buffers to override the syntax-ppss 
> logic via a newly introduced syntax-ppss-function variable.

Well, that was the point of the above function: in the former case, the
function body is changed to (mapcar 'car (nth 9 ppss)), and will stay
the same in the latter.