How to make Bison disregard tokens?

[Edsko de Vries]

Hi,

Is there any way I can make Bison ignore tokens when it can't deal with
them? I'll try to explain. For every completely blank line in the input,
my leexer generates a NOP token. In some situations the parser can deal
with this NOP token; in others it can't. Specifically, a NOP is a
statement:

statement ::= ... | NOP ;

but nothing else. For example, part of the grammar involves binary
expressions:

bin_op ::= expr OP expr ;

Now, when the user writes a binary expression, but inserts a blank line
somewhere, the parser might get something like "expr OP NOP expr", and I
get a syntax error, complaining about an unexpected NOP.

I could of course insert a "ignore_NOP" token in between every two
symbols in the grammar, where ignore_NOP is defined like

ignore_NOP ::= NOP | /* empty */ ;

but I'd rather not (the grammar is large, and not my own). If my
understanding of Bison's error recovery is correct, it doesn't help
either to add an "error" token to statement, because "expr OP NOP expr"
really isn't an error: it should execute the corresponding action rule
as if it had seen "expr OP expr".

In other words, I want the parser to deal with NOPs if it can (that is,
when it is expecting a statement), but completely ignore them otherwise.
I was looking at the generated error handling code, to see if I could
figure out if I could modify the error handler in such a way that if the
error is due to a NOP token, it simply ignores the NOP and continues,
but I'm not sure how to do it.

Any help would be appreciated!

Thanks,

Edsko de Vries
www.phpcompiler.org