Re: Finding out when a token is consumed

[Frank Heckenbach]

Hans Aberg wrote:

> At 13:33 +0200 2003/04/30, Frank Heckenbach wrote:
> >I have a bison parser, and I'd like to do something whenever a token
> >is consumed (used in a rule). More precisely, before an action is
> >run, something should be done for each terminal used in the rule.
> 
> [Please reply to/cc address@hidden
> 
> You are too unspecific about this "something" you want to do and why you
> want to do it.

The reason why I want to do it is that my language contains some
"directives" that can appear anywhere between tokens -- and
therefore cannot be part of the bison grammer (at least not without
huge extra work of allowing them in every single place in every
rule), so I handle them at the lexer level.

But doing it in the obvious way causes directives to be handled too
early when they appear before a lookahead token. Therefore I need to
know when the tokens are consumed. Then I can delay the directives
to the appropriate point (which has some more ramifications and
complications, but I think I can handle them, and they're not
directly related to Bison).

So basically, if I can get, say, a function called each time a token
is consumed (in the sense I described), I can do the rest (including
any necessary bookkeeping of tokens and directives as necessary).

> The Bison generated parser is tricky, because it may or may
> not need a lookahead token, depending what the grammars looks like.

I know -- that's just the problem I'm dealing with. If there was no
lookahead, I could do it in the lexer after getting a token. And if
there was always look-ahead, the check at the beginning of yylex (in
my example) would suffice. That's why I'm checking
yychar == YYEMPTY.

> The simplest way is to put as much work in the actions of the lexer/parser
> rules, and only do extra tweaking as necessary. For example, you might
> record the token names in the lexer rules, and then in the parser rule
> action you examine those names, doing that special thing you want. If the
> special thing is an environmental context, then one can let the rules
> perhaps via midrule actions set/pop those contexts in the form a stacked
> lookup table. Then the lexer rule can make a lookup in that table stack,
> and give that as $n value to the parser.

The problem is that it would have to be done for each action (if I
understand you correctly). The (ab)use of `YYLLOC_DEFAULT' as in my
example should actually work for my purposes. I just find it a bit
unelegant (but I'd rather do this than writing something in each
action).

Frank

-- 
Frank Heckenbach, address@hidden
http://fjf.gnu.de/
GnuPG and PGP keys: http://fjf.gnu.de/plan (7977168E)