[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
AW: [Grammatica-users] Tokenizer problem
From: |
HECKHAUSEN Ralf |
Subject: |
AW: [Grammatica-users] Tokenizer problem |
Date: |
Fri, 1 Jul 2005 14:46:54 +0200 |
Yes, it helps in this example. Let's take a more real example:
%header%
GRAMMARTYPE = "LL"
%tokens%
LETTER = <<[A-Z]>>
DIGIT = <<[0-9]>>
V = "V"
%productions%
INPUT = STATION_ID VARIATION;
STATION_ID = LETTER LETTER LETTER LETTER;
VARIATION = DIGIT DIGIT DIGIT V DIGIT DIGIT DIGIT;
Output for parsing "LOVW 123V345":
INPUT(2001)
STATION_ID(2002)
LETTER(1001): "L", line: 1, col: 1
LETTER(1001): "O", line: 1, col: 2
Error: in test: line 1:
unexpected token "V", expected <LETTER>
LOVW 123V345
^
What can I do about this?
Ralf
________________________________
Von: address@hidden im
Auftrag von Anant Mistry
Gesendet: Fr 01/07/2005 14:13
An: address@hidden
Betreff: Re: [Grammatica-users] Tokenizer problem
Ralf,
A possible solution would be to define
COMMAND = "GO "
That is a string of "GO" followed by the space character.
It makes your production
INPUT = COMMAND LETTER+;
Hope this helps
Anant
On Fri, 2005-07-01 at 10:48 +0200, HECKHAUSEN Ralf wrote:
I have a problem with the parser, or better the tokenizer. Below is a
simplified example.
%header%
GRAMMARTYPE = "LL"
%tokens%
LETTER = <<[A-Z]>>
COMMAND = "GO"
SPACE = " "
%productions%
INPUT = COMMAND SPACE LETTER+;
The input "GO SOUTH" is parsed correctly, but with "GO SOGO" I get a
parse error, because "GO" is not recognized as two letters anymore. It seems
that whenever letters are defined as tokens, they may not occur in other
context. But I am sure there is a way around this problem, can someone help
me?
Cheers,
Ralf
_______________________________________________
Grammatica-users mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/grammatica-users
--
Anant Mistry
Director of Infrastructure Technology
Information Handling Services
15 Inverness Way East
Englewood
CO 80112
USA
303 397 2569
Confidentiality Notice: The information in this e-mail may be confidential
and / or privileged. This e-mail is intended to be reviewed by only the
individual or organization named in the e-mail address. If you are not the
intended recipient, you are hereby notified that any review,
dissemination or copying of this e-mail and attachments, if any, or the
information contained herein, is strictly prohibited.
<<winmail.dat>>
- AW: [Grammatica-users] Tokenizer problem,
HECKHAUSEN Ralf <=