Re: [Grammatica-users] multiline in token definition

[Steffen Gaede]

> Hi Steffen,
> 
>> I'm writing the grammar for an Select-from-where string.
>> The Main Problem is, that the regExp at defining token not known about
>> exclude words (like command words):
>>
>> e.g.:
>>      FUNCTION_U =
>> <<(^((from|where|distinct|unique|or|and|not|group|order|by|having|as|asc|desc|on|like|between|in|count|min|max|avg|sum)[
>> ]*\())[a-z0-9]+[ ]*\(>>
>> doesn't work: It accepts then only FUNCTION_U with suffix of excluded
>> text like "minxxxx", "havingbbb" not "murx".
> 
> (Note: The reason it fails is that "^" works like "NOT" only inside of 
> "[...]".)

ok, that professing something.

> 
>> Ok, then I decided to build a combination, that has all literal
>> combination. 
> 
> I believe that either 1.5 or 1.6 has been modified such that the order in 
> which tokens are given represents priority. (To be exact, the first of all 
> longest matches will be chosen.) I.e., if you define
> 
>         KW_from = "from"
>         KW_where = "where"
>         ...
>         FUNCTION_U = <<[a-z0-9]+>>
> 
> then each keyword would be matched by both its specific definition as well as 
> the general one, but since the specific comes first, that will be chosen, and 
> you are fine.
> 
> (Also, please look at documentation and examples how whitespace should be 
> handled, e.g., "WHITESPACE = <<[ \t\n\r]+>> %ignore%".)

The Whitespace in the funtion is nessessary - because, if I've an
expression defined as:
        EXPRESSION = <<[a-z0-9_]+>>
and a function with
        MIN        = "min"
and a symbol with
        LEFT_PAREN = "("
        RIGHT_PAREN = ")"
        DOT     = "."

and my SFW has "select min(Person.Salary) ..."
he searches for "min(" instead of "min" "leftparen" although I've defined:
        FUNCTION1 = MIN|MAX;
        S1 = FUNCTION1 "(" Expression ["." Expression] ")";

(the same with "min (Person.Salary) ..." - he search for "min ("
So it does work only with:
        MIN     = <<min[ ]*\(>>
and
        S1 = FUNCTION1 Expression ["." Expression] ")";


> Please try if the above works. If you have 1.5 and I am wrong and the new 
> priority determination is only included in 1.6, then the old one should apply 
> which says that literals ("...") are preferred over regexes (<<...>>), so it 
> should work in your case, too.
That was my first problem, before I've tried the full-word exclusion.

Steffen.