Re: improving yysyntax_error()

[Christian Schoenebeck]

Es geschah am Thursday, 21. June 2007 16:08 als Hans Aberg schrieb:
> No, in this model, the lexer matches patterns as usual, only when the
> match has been made, returns the multibyte character byte by byte.

Doesn't make a difference. The problem for the UTF-8 part is still the same, 
no matter if you implement the non-UTF-8 part with an external lexer or not.

> > For supporting UTF-8 characters, you could
> > define "atomic" grammar rules, like:
> >
> > UTF8_CAPITAL_PI : '\316' '\240' ;
> >
> > UTF8_CAPITAL_OMEGA : '\316' '\231' ;
> >
> > and use the suggested new declaration keyword "%atomic" like:
> >
> > %atomic UTF8_CAPITAL_PI UTF8_CAPITAL_OMEGA
> >
> > to tell bison the right hand side of those UTF-8 character rules
> > (that is
> > their byte sequene) is too trivial / unteresting to be shown in
> > errors and
> > yysyntax_error() would i.e return:
> >
> > "syntax error, unexpected 'UTF8_CAPITAL_OMEGA', expecting
> > 'UTF8_CAPITAL_PI"
> >
> > instead of:
> >
> > "syntax error, unexpected '\231', expecting '\240"
> >
> > The latter would be completely useless and confusing for regular
> > users.
>
> The stuff above isn't needed, except for the generation of error
> messages. So how do you intend to implement your %atomic construct?

With "The stuff above" you probably just mean the "%atomic" line. The rest 
would be needed for implementing UTF-8 support. And yes, for now it would 
just be useful for better error and debug messages. Another future 
application would be integrated type completion support within the bison 
skeleton parser.

CU
Christian