Re: RFC: enum instead of #define for tokens

[Hans Aberg]

At 07:46 -0800 2002/04/05, John W. Millaway wrote:
>> If Flex adapts the same mangling scheme, then `...' within actions should
>> be mangled into C/C++ labels. Thus,
>>   "!=" { return `!=' }
>> might be legal. Or one may even has that as default.
>
>I doubt it will ever fly. What about '\012', which is legal C, but should not
>be "mangled"? One thing about flex, it doesn't mess with your C code, and
>that's probably a good thing to preserve.

I'm not sure what you mean here: `...' and '...' are syntactically
different; thus, one can choose whatever convention one likes for `...'.

The scheme I made use of, first has an encoding for `...' that ensures one
can produce any binary string of any length. Then I made a specific scheme
for mangling it into C/C++ labels. It would be easy to change the mangling
scheme to labels of any other language; then I do not have to change the
encoding scheme into binary strings.

>> But
>>   . { return yytext[0]; }
>> would not work. It is still possible to reserve this character range, so
>> that Bison can issue the proper error message.
>
>But remember we're looking ahead to unicode.

As for the scheme above, it easy to extend it to Unicode: First have a way
to indicate that the `..' string should be translate as binary Unicode
string; then have a mangling scheme for Unicode strings into C/C++
identifiers.

I think that Akim would like to prohibit
     . { return yytext[0]; }
at least for Unicode. But I think it might be used as a convenient way to
produce parser errors for sporadic characters; then the parser can use its
error recovery system to handle it.

  Hans Aberg