Re: Inserting extra tokens

[Hans Aberg]

On 24 Aug 2006, at 08:00, Erik Sandberg wrote:

> I found a solution to my problem: If a function takes two  
> arguments, then the
> lexer generates three tokens for the function name:
> \function
> =>
> FUNCTION EXPECT_ARG EXPECT_ARG
> and then the grammar for the argument list looks like:
> argument_list: /*nothing*/ | EXPECT_ARG argument_list argument;
>
> which works perfectly.

The idea of letting the lexer to insert a token has been used, for  
example, in the .l grammar of Bison self.

The advantage is that it simplifies the parser (.y) grammar, but it  
may result in a complicated lexer (.l) grammar. So if this becomes a  
problem, a GLR grammar might be attempted.

  Hans Aberg