fix for error location handling in Yacc parsers

[Paul Eggert]

I installed the following patch to handle a problem that Paul
Hilfinger noted last month.  The GLR parser fails the revised test
case, so I've left the GLR variant of the test case commented out
(with a FIXME comment).

2002-12-27  Paul Eggert  <address@hidden>

        Fix a bug where error locations were not being recorded correctly.
        This problem was originally reported by Paul Hilfinger in
        <http://mail.gnu.org/pipermail/bug-bison/2002-November/001901.html>.

        * data/yacc.c (yyparse): New local var yylerrsp, to record the
        top of the location stack's error locations.
        (yyerrlab): Set it.  When discarding a token, push its location
        onto yylerrsp so that we don't lose track of the error's end.
        (yyerrlab1): Now is only the target of YYERROR, so that we can
        properly record the location of the action that failed.  For GCC
        2.93 and later, insert an __attribute__ ((__unused__)) to avoid
        GCC warning about yyerrlab1 being unused if YYERROR is unused.
        (yyerrlab2): New label, which yyerrlab now falls through to.
        Compute the error's location by applying YYLLOC_DEFAULT to
        the locations of all the symbols that went into the error.
        * doc/bison.texinfo (Location Default Action): Mention that
        YYLLOC_DEFAULT is also invoked for syntax errors.
        * tests/actions.at (_AT_CHECK_PRINTER_AND_DESTRUCTOR):
        Error locations include the locations of all the tokens that were
        discarded, not just the last token.

Index: data/yacc.c
===================================================================
RCS file: /cvsroot/bison/bison/data/yacc.c,v
retrieving revision 1.46
diff -p -u -r1.46 yacc.c
--- data/yacc.c 26 Dec 2002 08:27:09 -0000      1.46
+++ data/yacc.c 28 Dec 2002 08:32:57 -0000
@@ -54,7 +54,7 @@ m4_ifset([b4_parse_param], [b4_c_args(b4
 
 # b4_lex_param
 # ------------
-# Accumule in b4_lex_param all the yylex arguments.
+# Accumulate in b4_lex_param all the yylex arguments.
 # b4_lex_param arrives quoted twice, but we want to keep only one level.
 m4_define([b4_lex_param],
 m4_dquote(b4_pure_if([[[[YYSTYPE *]], [[&yylval]]][]dnl
@@ -767,7 +767,8 @@ b4_c_function_def([yyparse], [int], b4_p
 [[  /* The location stack.  */
   YYLTYPE yylsa[YYINITDEPTH];
   YYLTYPE *yyls = yylsa;
-  YYLTYPE *yylsp;]])[
+  YYLTYPE *yylsp;
+  YYLTYPE *yylerrsp;]])[
 
 #define YYPOPSTACK   (yyvsp--, yyssp--]b4_location_if([, yylsp--])[)
 
@@ -1072,6 +1073,8 @@ yyerrlab:
        yyerror (]b4_yyerror_args["syntax error");
     }
 
+]b4_location_if([  yylerrsp = yylsp;])[
+
   if (yyerrstatus == 3)
     {
       /* If just tried and failed to reuse lookahead token after an
@@ -1095,17 +1098,34 @@ yyerrlab:
       YYDSYMPRINTF ("Error: discarding", yytoken, &yylval, &yylloc);
       yydestruct (yytoken, &yylval]b4_location_if([, &yylloc])[);
       yychar = YYEMPTY;
+]b4_location_if([      *++yylerrsp = yylloc;])[
     }
 
   /* Else will try to reuse lookahead token after shifting the error
      token.  */
-  goto yyerrlab1;
+  goto yyerrlab2;
 
 
 /*----------------------------------------------------.
 | yyerrlab1 -- error raised explicitly by an action.  |
 `----------------------------------------------------*/
 yyerrlab1:
+
+  /* Suppress GCC warning that yyerrlab1 is unused when no action
+     invokes YYERROR.  */
+#if defined (__GNUC_MINOR__) && 2093 <= (__GNUC__ * 1000 + __GNUC_MINOR__)
+  __attribute__ ((__unused__))
+#endif
+
+]b4_location_if([  yylerrsp = yylsp;
+  *++yylerrsp = yyloc;])[
+  goto yyerrlab2;
+
+
+/*---------------------------------------------------------------.
+| yyerrlab2 -- pop states until the error token can be shifted.  |
+`---------------------------------------------------------------*/
+yyerrlab2:
   yyerrstatus = 3;     /* Each real token shifted decrements this.  */
 
   for (;;)
@@ -1140,7 +1160,8 @@ yyerrlab1:
   YYDPRINTF ((stderr, "Shifting error token, "));
 
   *++yyvsp = yylval;
-]b4_location_if([  *++yylsp = yylloc;])[
+]b4_location_if([  YYLLOC_DEFAULT (yyloc, yylsp, (yylerrsp - yylsp));
+  *++yylsp = yyloc;])[
 
   yystate = yyn;
   goto yynewstate;
Index: doc/bison.texinfo
===================================================================
RCS file: /cvsroot/bison/bison/doc/bison.texinfo,v
retrieving revision 1.95
diff -p -u -r1.95 bison.texinfo
--- doc/bison.texinfo   24 Dec 2002 07:43:43 -0000      1.95
+++ doc/bison.texinfo   28 Dec 2002 08:32:59 -0000
@@ -3186,17 +3186,23 @@ Actually, actions are not the best place
 locations are much more general than semantic values, there is room in
 the output parser to redefine the default action to take for each
 rule.  The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
-matched, before the associated action is run.
+matched, before the associated action is run.  It is also invoked
+while processing a syntax error, to compute the error's location.
 
 Most of the time, this macro is general enough to suppress location
 dedicated code from semantic actions.
 
 The @code{YYLLOC_DEFAULT} macro takes three parameters.  The first one is
-the location of the grouping (the result of the computation).  The second one
-is an array holding locations of all right hand side elements of the rule
-being matched.  The last one is the size of the right hand side rule.
+the location of the grouping (the result of the computation).  When a
+rule is matched, the second parameter is an array holding locations of
+all right hand side elements of the rule being matched, and the third
+parameter is the size of the rule's right hand side.  When processing
+a syntax error, the second parameter is an array holding locations of
+the symbols that were discarded during error processing, and the third
+parameter is the number of discarded symbols.
 
-By default, it is defined this way for simple @acronym{LALR}(1) parsers:
+By default, @code{YYLLOC_DEFAULT} is defined this way for simple
address@hidden(1) parsers:
 
 @example
 @group
@@ -3491,7 +3497,7 @@ should use @code{$$} to designate the se
 (@pxref{Parser Function, , The Parser Function @code{yyparse}}).
 
 @strong{Warning:} as of Bison 1.875, this feature is still considered as
-experimental, as there was not enough users feedback.  In particular,
+experimental, as there was not enough user feedback.  In particular,
 the syntax might still change.
 @end deffn
 
Index: tests/actions.at
===================================================================
RCS file: /cvsroot/bison/bison/tests/actions.at,v
retrieving revision 1.26
diff -p -u -r1.26 actions.at
--- tests/actions.at    27 Nov 2002 18:31:27 -0000      1.26
+++ tests/actions.at    28 Dec 2002 08:33:00 -0000
@@ -157,7 +157,7 @@ yylex (void)
 {
   static char const input[] =
     {
-      /* Exericise the discarding of stack top and input until `error'
+      /* Exercise the discarding of stack top and input until `error'
          can be reduced.  */
       'x', 'x', 'x', 'x', 'x', 'x', ';',
 
@@ -479,7 +479,7 @@ Freeing token 'x' (address@hidden)
 sending: 'x' (address@hidden)
 Freeing token 'x' (address@hidden)
 sending: ';' (address@hidden)
-line (address@hidden): error (@50-59) ';' (address@hidden)
+line (address@hidden): error (@0-59) ';' (address@hidden)
 sending: 'x' (address@hidden)
 thing (address@hidden): 'x' (address@hidden)
 sending: 'x' (address@hidden)
@@ -495,7 +495,7 @@ sending: 'y' (address@hidden)
 sending: EOF
 Freeing nterm line (address@hidden)
 Freeing nterm line (address@hidden)
-Freeing nterm line (address@hidden)
+Freeing nterm line (address@hidden)
 Parsing FAILED.
 ]])
 
@@ -513,5 +513,5 @@ m4_define([AT_CHECK_PRINTER_AND_DESTRUCT
 
 AT_CHECK_PRINTER_AND_DESTRUCTOR()
 
-# This test fails, but the test case is questionable.
+# FIXME.  This test case fails.
 #AT_CHECK_PRINTER_AND_DESTRUCTOR([%glr-parser])