Re: How to decide what to put in the lexer and the grammar respectively?

[Akim Demaille]

> Le 18 févr. 2019 à 00:10, Hans Åberg <address@hidden> a écrit :
> 
> 
>> On 17 Feb 2019, at 23:10, Peng Yu <address@hidden> wrote:
>> 
>> This lexical tie-in creates feedback from the parser to the lexer. So
>> the lexer cannot be tested standalone.
>> 
>> But the principle of separating lexer and parser is to make parser
>> builtin on top of the parser. Is there something that can avoid the
>> feedback yet still allow context-dependent parsing? Alternatively, how
>> to just testing the lexer without having to get the parser involved?
> 
> The LARL(1) that Bison uses is for context free grammars only, so contexts 
> must involve switches somehow.

I don't think Peng was referring to context-sensitivity here in the
technical sense.  He just means "lexical context depending on the
current state of the parser", which is still CFG.  Let's not confuse
people by referring to something unrelated.