Re: [Grammatica-users] multiline in token definition

[Oliver Gramberg]

Hi Steffen,

> I'm writing the grammar for an Select-from-where string.
> The Main Problem is, that the regExp at defining token not known about
> exclude words (like command words):
> 
> e.g.:
>       FUNCTION_U =
> <<(^((from|where|distinct|unique|or|and|not|group|order|by|having|as|asc|desc|on|like|between|in|count|min|max|avg|sum)[
> ]*\())[a-z0-9]+[ ]*\(>>
> doesn't work: It accepts then only FUNCTION_U with suffix of excluded
> text like "minxxxx", "havingbbb" not "murx".

(Note: The reason it fails is that "^" works like "NOT" only inside of "[...]".)

> Ok, then I decided to build a combination, that has all literal
> combination. 

I believe that either 1.5 or 1.6 has been modified such that the order in which 
tokens are given represents priority. (To be exact, the first of all longest 
matches will be chosen.) I.e., if you define

        KW_from = "from"
        KW_where = "where"
        ...
        FUNCTION_U = <<[a-z0-9]+>>

then each keyword would be matched by both its specific definition as well as 
the general one, but since the specific comes first, that will be chosen, and 
you are fine.

(Also, please look at documentation and examples how whitespace should be 
handled, e.g., "WHITESPACE = <<[ \t\n\r]+>> %ignore%".)

This is much shorter and clearer and easier to understand and maintain than 
trying to press everything into a big regex. I am afraid that there will be 
other locations in the grammar where you would have to repeat this, and then it 
would get really ugly...

Please try if the above works. If you have 1.5 and I am wrong and the new 
priority determination is only included in 1.6, then the old one should apply 
which says that literals ("...") are preferred over regexes (<<...>>), so it 
should work in your case, too.

Regards
Oliver
-- 
Diese Signatur besteht zu 100% aus wiederverwendeten Pixeln.


Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de