Re: improving yysyntax_error()

[Hans Aberg]

It happened on 21 Jun 2007, at 16:43, that Christian Schoenebeck wrote:

> Es geschah am Thursday, 21. June 2007 16:08 als Hans Aberg schrieb:
> > No, in this model, the lexer matches patterns as usual, only when the
> > match has been made, returns the multibyte character byte by byte.
>
> Doesn't make a difference. The problem for the UTF-8 part is still  
> the same,
> no matter if you implement the non-UTF-8 part with an external  
> lexer or not.

I do not see there is any problem: the lexer returns a sequence of  
bytes, which the parser identifies, a capability it alreday has, so  
one adds some syntactic sugar for it.

> > > For supporting UTF-8 characters, you could
> > > define "atomic" grammar rules, like:
> > >
> > > UTF8_CAPITAL_PI : '\316' '\240' ;
> > >
> > > UTF8_CAPITAL_OMEGA : '\316' '\231' ;
> > >
> > > and use the suggested new declaration keyword "%atomic" like:
> > >
> > > %atomic UTF8_CAPITAL_PI UTF8_CAPITAL_OMEGA
> > >
> > > to tell bison the right hand side of those UTF-8 character rules
> > > (that is
> > > their byte sequene) is too trivial / unteresting to be shown in
> > > errors and
> > > yysyntax_error() would i.e return:
> > >
> > > "syntax error, unexpected 'UTF8_CAPITAL_OMEGA', expecting
> > > 'UTF8_CAPITAL_PI"
> > >
> > > instead of:
> > >
> > > "syntax error, unexpected '\231', expecting '\240"
> > >
> > > The latter would be completely useless and confusing for regular
> > > users.
> >
> > The stuff above isn't needed, except for the generation of error
> > messages. So how do you intend to implement your %atomic construct?
>
> With "The stuff above" you probably just mean the "%atomic" line.

How should %atomic be implemented?

> The rest
> would be needed for implementing UTF-8 support.

No, as sequences of bytes can be put directly into the rules.

> And yes, for now it would
> just be useful for better error and debug messages.

So here I want to know how it should be implemented.

> Another future
> application would be integrated type completion support within the  
> bison
> skeleton parser.

I cannot parse this. Please elaborate.

  Hans Aberg