[Grammatica-users] Tokenizer problem

[HECKHAUSEN Ralf]

I have a problem with the parser, or better the tokenizer. Below is a simplified example.

%header%

GRAMMARTYPE = "LL"

%tokens%
LETTER = <<[A-Z]>>
COMMAND = "GO"
SPACE = " "

%productions%

INPUT = COMMAND SPACE LETTER+;

The input "GO SOUTH" is parsed correctly, but with "GO SOGO" I get a parse error, because "GO" is not recognized as two letters anymore. It seems that whenever letters are defined as tokens, they may not occur in other context. But I am sure there is a way around this problem, can someone help me?

 

Cheers,

Ralf