Erroneous YYBACKUP skeleton

[David Kastrup]

Hi,

digging around on the web, I found an example of a failing YYBACKUP test
case in <URL:http://old.nabble.com/-PATCH--Update-TODO.-p20418035.html>
which has made it into the Bison TODO.  Further digging revealed
<URL:http://www.mail-archive.com/address@hidden/msg03249.html>.

Combining the information from both, the problem definitely appears to
be that YYBACKUP currently (as of bison 2.4.1, the version coming with
Ubuntu 11.10) does not correctly revert the state after popping the
stack.

I have tried browsing Savannah, but it does not allow me to call the
equivalent of "git grep YYBACKUP" on the repository, and the
organisation of the skeleton source is so clever that I can't locate the
file containing the definition of YYBACK myself.  The TODO entry,
however, still is there.

So I can report just my findings: The following changed macro does
appear to do the trick:

#define YYBACKUP(Token, Value)                                  \
do                                                              \
  if (yychar == YYEMPTY)                                \
    {                                                           \
      yychar = (Token);                                         \
      yylval = (Value);                                         \
      yytoken = YYTRANSLATE (yychar);                           \
      YYPOPSTACK (yylen);                                               \
      yystate = *yyssp; \
      goto yybackup;                                            \
    }                                                           \
  else                                                          \
    {                                                           \
      yyerror (YY_("syntax error: cannot back up")); \
      YYERROR;                                                  \
    }                                                           \
while (YYID (0))
}

As compared to the version in the default skeleton, there are two
changes.  The first is an obvious enhancement: the "if" does not check
yylen, but YYPOPSTACK takes its argument from it.  That way, rules
reducing more than one token can also be backed up.

The actual bug fix is the addition of the line yystate = *yyssp which
makes the proposed example from the TODO (which is still there in
current master) actually work.

It is a pity that I won't be able to use YYBACKUP for my current project
(Lilypond), but it seems worthwhile to have it fixed eventually.  I have
not checked that the enhancement for reducing rules with more than one
token does actually work.  It seems like an obvious change, but I don't
really understand Bison enough to be sure.

-- 
David Kastrup