Re: Unicode Bison

[Hans Aberg]

At 11:53 -0400 2002/04/11, Frédéric Morin wrote:
>is there a unicode bison version ?

Nope, if you by that mean a version which can detect errors when the lexer
turns over illegal Unicode codes, and by which Unicode characters can be
included in the grammar (.y) file. (If you do not hand over Unicode
character coeds to the parser, you should have no problem though.)

> or what can I do to transform flex and
>bison to be compliant with unicode ?

As for Flex, stay tuned at
  Help-flex mailing list
  address@hidden
  http://mail.gnu.org/mailman/listinfo/help-flex

There is a Flex patch called "Unicode"
  ftp://ftp.lauton.com/pub/flex-2.5.4-unicode-patch.tar.gz
But what I think what it does is merely patching for use with wchar's,
which it assumes are 16 bit. Current Unicode uses 21 bits or something, and
the array tables becomes very large, so for that to work, one would have to
use compressed tables.

As for Bison, the hardest part might be to persuade the current Bison
maintainers. :-) But one the first level, one might make Bison generated
parsers be able to detect illegal Unicode characters; this is easy. Then
next level would be to allow Unicode characters in the grammar (.y) file;
then one will have to use compressed tables, as in the case of the Flex
generated lexers.

  Hans Aberg