Dealing with errors generated while lexing

[Nikolai Weibull]

Hi!

Is there a simple way for yylex to tell the parser to YYABORT?

I’ve tried to figure out how to deal with errors generated while
lexing.  Say that I hit an out-of-memory condition while lexing and
want to abort the parse.  I see no easy, general way of reaching a
point where I can YYABORT.  It’s manageable for tokens that have a
value, as you can simply check for an error value, for example, NULL,
but I can’t seem to come up with a good solution for non-value tokens.
 I would like to be able to return an error token from yylex, but
there doesn’t seem to be any provisions for doing so.  I guess one
could return a token that’s not allowed anywhere, but that would
generate a syntax error, which isn’t the kind of handling that I want.
 One could further try to handle this error token, but then one would
have to add a rule for that case in a lot of places.  You could add a
value to the token as well and check for an error value, but it won’t
work for rules such as

a: TOKEN b;

as you won’t be able to check if TOKEN has the error value until b has
been seen and the action has been entered.

I was also thinking of using setjmp in an %initial-action and then
longjmping to it if an error occured and from there YYABORTing, but if
I understand my ISO C, yychar, yylen, and other auto variables that
the yyreturn code depends upon will have indeterminate values, meaning
that cleanup might not be performed correctly.  I noticed that the glr
parser uses this kind of code to get out of out-of-memory conditions,
but as far as I can tell, its code is similarly susceptible to using
indeterminate values for these variables.  Am I wrong?

Thanks!