Re: Enum for token '0', EOF

[Joel E. Denny]

On Wed, 12 Jul 2006, Joel E. Denny wrote:

> I went ahead.  As far as I can tell, all the skeletons now (after my 
> uncommitted patch below) consistently pop EOF one time upon a successful 
> parse.

I'd like to commit this patch.  I'll do so in a few days if there are no 
objections.

> I don't understand why the parser didn't clear the lookahead when it 
> shifted EOF (actually, glr.c did clear it but only in non-deterministic 
> operation).  That meant that, upon a parse failure, the parser didn't 
> destroy an unshifted EOF lookahead for fear that it might have already 
> shifted it.  I removed this behavior, and everything seems fine.  Is that 
> ok?

I'm a little worried about this change because I don't know the reason for 
the current behavior.  Does anyone see a problem?  Here are the relevant 
changes in yacc.c:

> Index: data/yacc.c
> ===================================================================
> RCS file: /sources/bison/bison/data/yacc.c,v
> retrieving revision 1.150
> diff -p -u -r1.150 yacc.c

> @@ -1224,9 +1224,8 @@ yybackup:
>    /* Shift the lookahead token.  */
>    YY_SYMBOL_PRINT ("Shifting", yytoken, &yylval, &yylloc);
>  
> -  /* Discard the shifted token unless it is eof.  */
> -  if (yychar != YYEOF)
> -    yychar = YYEMPTY;
> +  /* Discard the shifted token.  */
> +  yychar = YYEMPTY;
>  
>    yystate = yyn;
>    *++yyvsp = yylval;

Above is where a YYEOF lookahead wasn't cleared.

> @@ -1461,7 +1457,7 @@ yyexhaustedlab:
>  #endif
>  
>  yyreturn:
> -  if (yychar != YYEOF && yychar != YYEMPTY)
> +  if (yychar != YYEMPTY)
>       yydestruct ("Cleanup: discarding lookahead",
>                yytoken, &yylval]b4_locations_if([, &yylloc])[]b4_user_args[);
>    /* Do not reclaim the symbols of the rule which action triggered

Above is where a YYEOF lookahead wasn't destroyed.

The full patch is here:

  http://lists.gnu.org/archive/html/help-bison/2006-07/msg00023.html

Joel