bug#23019: parse-partial-sexp doesn't output the full state needed for its continuance.

[Stefan Monnier]

>> It sounds to me like there's a chance it's actually incomplete (e.g.
>> it doesn't address the similar problem when the "last character
>> scanned" is an end of a string which also happens to be a valid
>> first-char of a comment-starter), and even if it isn't, it "feels
>> ad-hoc" to me.
> Now even I wouldn't have come up with that end-of-string scenario.  ;-)

I don't work in embedded systems, but Coq/Agda's total functions force
you to consider all possible cases.

> Such a scenario is presumably one reason why, in scan_sexps_forward, two
> character comment delimiters are handled before strings.

It doesn't handle the exact same situation, but it's closely related
indeed.

>> - change element 10 so it's nil if the last char was an "end of
>> something".  Another way to look at it, is that the element 10 should
>> only be non-nil if the "next lexeme" might start on that
>> previous character.

> I've tried this, and it's somewhat ugly.  Setting the "previous_syntax"
> to nil is also needed for the asterisk in "/*".  The nil would appear to
> mean "the syntactic value of the last character has already been used
> up".  So the "previous_syntax" is nil in the most interesting cases.  It
> also feels somewhat ad-hoc.

> How about this idea: element 10 will record the syntax of the previous
> character ONLY when it is potentially the first character of a two
> character comment delimiter, otherwise it'll be nil.  At least that's
> being honest about what the thing's being used for.

IIUC the only difference between what I (think I) suggested and what
you're proposing is that you want to return nil for the "prev is
backslash" whereas I was suggesting to return non-nil in that case.
[ AFAIK the only two-char elements we handle so far as the comment
delimiters and the backslash escapes.  ]
Do I understand this right?

> It would appear to be, yes.  We really can't get rid of element 5,
> though, because there will surely be code out there that uses it.  But
> if I change element 10 as outlined above, element 5 will no longer be
> redundant.

I'd even be tempted to re-use element 5, although it might
conceivably break some code out there.

But even if we don't re-use element 5, I would actually much prefer to
render element 5 redundant.


        Stefan