Re: How to decide what to put in the lexer and the grammar respectively?

[Akim Demaille]

Hi Peng!

> Le 17 févr. 2019 à 23:10, Peng Yu <address@hidden> a écrit :
> 
> This lexical tie-in creates feedback from the parser to the lexer. So
> the lexer cannot be tested standalone.

Well, yes, it can, but that's not as elegant, agreed.

> But the principle of separating lexer and parser is to make parser
> builtin on top of the parser. Is there something that can avoid the
> feedback yet still allow context-dependent parsing?

Nothing I can think of ATM.  Except scannerless parsing, that we don't
have.

> Alternatively, how
> to just testing the lexer without having to get the parser involved?

Let your tests deal with the context.  But you certainly have thought
about that, so I might have misunderstood your question.