Re: Syntax tables for multiple modes [was: bug#22983: syntax-ppss returns wrong result.]

[Stefan Monnier]

> To be frank, the issue of parse-partial-sexp is fading because modes
> are now using syntax-ppss more extensively.

Aha!  So we agree: there's no reason to worry about parse-partial-sexp,
since at this point pretty much all modes rely on syntax-ppss (other
than CC-modes, obviously).

> Most of the problems with parse-partial-sexp from the past are now
> internalized in condition-case inside syntax-ppss. That condition-case
> is triggered very often (almost always) from inside polymode
> chunk narrowing.

Right.  But I don't see it as a problem in parse-partial-sexp, rather
than a problem in syntax.el.

>> Because you need it for *everything* that looks at the buffer.
>> Why should parse-partial-sexp be different from (say) scan-sexps?
> I think parse-partial-sexp, syntax-ppss and maybe some others, are special in
> the sense that in order to return a correct value they need to be aware of the
> whole buffer. I don't see this as an inconsistency but I might be too naive.

scan-sexps will complain about unmatched parens and things like that if
it bumps into point-min/max.  Same for re-search-*.  I think you've just
been too often exposed to the use of (parse-partial-sexp 1 ...) where
the resulting signal bites you right away, whereas many other functions
won't signal an error and will instead do *something* (which may not
always be incorrect, but may often enough still result in acceptable
behavior).

> c-mode provides an example. I don't remember where exactly and how but
> it has to do with but c-before-context-fl-expand-region and
> c-state-semi-safe-place because I am advising these two
> functions currently.

CC-modes is definitely a very special case here.
We should aim to limit the amount of changes in most major modes, so
better not pay too much attention to cc-mode from that point of view.

> The logic is roughly like this, c-mode engine doesn't respect
> font-lock-dont-widen, widens stuff in some of it's functions, then
> calls its parsing, gets back some points outside font-lock range and
> blows when trying to access those points from narrowed region.

Sounds like a problem in cc-mode, which will require changes in
cc-mode.  The generic code shouldn't worry about that.

> I was not collecting these cases carefully but I will start doing it and will
> get with more concrete examples in the following weeks.

Thanks.

> Another directions of problems is syntax-propertize.  It can be called
> with POS outside of current narrowed region.  Particularly from
> internal--syntax-propertize.

That would definitely be an error, so if you bump into such a case
please report it.

>> Replace all your widen calls with calls to `prog-widen' and you get
>> the same result (since (nth 1 prog-indentation-context) is basically
>> another name for your hard-widen-limit).  So I don't think prog-widen
>> is that very complicated.
> It's not but you have to enforce that in all known modes.

I prefer to say that "any major mode which wants to play with the new
snazzy multi-mode feature needs to be adjusted (e.g. with prog-widen)".
It's perfectly fine if some major modes don't play along correctly until
they're fixed.

"Try to get multi-mode working without touching anyone's code"
(e.g. using advice) is great, but we already have packages which do that.

>> Note that "is more general" here means that the major mode's function has to
>> handle more cases, so it would seem to fundamentally require more work on the
>> major mode's side.

> I don't agree. Work must be done only in the generic multi-mode engines
> (mmm-mode, polymode etc).

The "is more general" I was quoting was talking about the ways the
generic code can call the major-mode-specific code.  If this is more
generic, it means the major-mode-specific code needs to handle more
situations (e.g. STRING-BEFORE appearing not just at the beginning of
a chunk).

> Other modes should re-use that generic infrastructure, or even better,
> do nothing, and leave to someone else to define a new polymode with
> host chunk being *the* mode. That every mode with basic needs for
> inner sub-modes tries to re-invent the wheel is a dead end.

I don't understand: every major mode's indentation code will have to pay
attention to the STRING-BEFORE/AFTER that it receives from the generic
code and will have to do something with it (it can ignore it but at the
cost of sub-optimal results).  And AFAIK this can only be done by the
major mode's code, not the generic mode's code.
[ I feel like I must be missing something.  ]


        Stefan