Re: How to decide what to put in the lexer and the grammar respectively?

[Hans Åberg]

> On 18 Feb 2019, at 06:44, Akim Demaille <address@hidden> wrote:
> 
>> Le 18 févr. 2019 à 00:10, Hans Åberg <address@hidden> a écrit :
>> 
>>> On 17 Feb 2019, at 23:10, Peng Yu <address@hidden> wrote:
>>> 
>>> This lexical tie-in creates feedback from the parser to the lexer. So
>>> the lexer cannot be tested standalone.
>>> 
>>> But the principle of separating lexer and parser is to make parser
>>> builtin on top of the parser. Is there something that can avoid the
>>> feedback yet still allow context-dependent parsing? Alternatively, how
>>> to just testing the lexer without having to get the parser involved?
>> 
>> The LARL(1) that Bison uses is for context free grammars only, so contexts 
>> must involve switches somehow.
> 
> I don't think Peng was referring to context-sensitivity here in the
> technical sense.  He just means "lexical context depending on the
> current state of the parser", which is still CFG.  Let's not confuse
> people by referring to something unrelated.

Subsequent discussions might clarify that.