Re: Enum for token '0', EOF

[Joel E. Denny]

On Sun, 9 Jul 2006, Akim Demaille wrote:

> I'd like to see its
> destructor called, just as in glr.

I'll implement whatever we decide for glr.c.  Maybe someone else can 
handle yacc.c?

> As a matter of fact, I have considered adding a BOF in complement of
> EOF: it seems that this token would allow to clean some minor details
> such as the initial yylval and yylloc.  It also allows to have a
> full definition of state 0: BOF is its accessing symbol.  I never
> went to the bottom of this, it might be dumb :)

glr.c does something like this, but it uses EOF as both BOF and EOF.  I 
haven't explored this in detail in a while, but I recall that it's the 
reason for the double pop of EOF.

> > I'm also working on the per-type and default %destructor and %printer.
> 
> For the records, I originally refrained from using per type destructor
> and printer, because I don't like the explicit handling of the union
> field name.  But I agree it would dramatically simplify today's uses.

I think we'll be able to extend it to handle your new %type as well.

> > I'm trying to figure out if they should be used for the end token.  If the
> > answer to the above question is no, the answer to this one is easy.
> 
> Honestly, as an answer to your question, another question: is there a reason
> *not* to allow it.  I didn't see any.

As a reminder, the default %destructor/%printer would be declared like 
this:

  %destructor { free ($$); }
  %printer { fprintf (yyouput, "%d", $$); }

That is, no symbols or types would be listed.  Bison would associate the 
default %destructor/%printer with all symbols for which no other 
%destructor/%printer is declared.  This feature would make the most sense 
when the user does #define YYSTYPE char *, or something like that.  It 
could be slightly dangerous with %union if it doesn't make sense for some 
fields, but I think we should trust the user to get it right and give him 
this flexibility.

The problem would be when Bison associated the default %destructor with 
the end token unexpectedly.  Bison would warn that $2 is unused in the 
generated rule 0.  Also, if the default %destructor/%printer were invoked 
for the end token, it might try to access uninitialized memory.  So, I 
originally figured we could just eliminate this trouble by saying 
%destructor/%printer isn't permitted for the end token at all.

However, maybe there's a better fix.  I'm not sure exactly what the 
Bison-generated parser does now, but I think it does not perform the 
reduction for rule 0.  Maybe it should?  Reducing rule 0 would = a 
successful parse.  Bison could automatically add an action to rule 0 that 
invokes any %destructor for the start symbol and any %destructor for the 
end token.  Might be a clean way to block the "unused value" warnings.

If the user doesn't declare the end token, then it's called $end, and 
Bison wouldn't associate any %destructor/%printer (not even the default) 
with it.  Thus, Bison wouldn't invoke any %destructor on it in the rule 0 
action.  In general, it'll prevent uses of the end token's uninitialized 
$$ and @$.

What do you think?

> > > > > but I need to signal it in my hand written tokenizer(and it is messy
> > > > > to
> > > > > cast
> > > > > 0 to the yytokentype enum).
> > > > 
> > > > I'm guessing this feature was originally intended for no more than just
> > > > that (and giving the end token a user-friendly string alias).
> > > 
> > > No it was not.  The original motivation was truly to provide a string
> > > alias while sticking to the existing interface for the other tokens.
> > 
> > Ok, but I guess you don't object to either usage?
> 
> I'm not sure what you are referring to.  The use of END in the
> rules themselves?  I don't remember how my friend used it, but it
> seemed smart, so I'd say I don't object.

Sorry, I meant Frans' usage as described above: to avoid the cast.  I only 
asked because I thought you might be saying that's not what the feature 
was originally intended for.

Joel