Re: RFC: enum instead of #define for tokens

[Paul Eggert]

> From: Akim Demaille <address@hidden>
> Date: 02 Apr 2002 16:34:35 +0200
> 
> Does POSIX mandate something?

POSIX 1003.1-2001 requires that the header file must contain #define
directives for each token code assigned by yacc.  The symbols must be
defined to be "numbers".  POSIX also talks about other things being in
the header file (#line directives, YYSTYPE, yylval), but it does not
say that this is an exhaustive list of what is in the header file, so
I think it's be OK to have an enum declaration too.  POSIX does
require that the header file be suitable as input for a C99 compiler.

Some older K&R compilers do not have enum, or have problems using them.

> From: Miles Bader <address@hidden>
> Date: 02 Apr 2002 23:45:03 +0900
> 
>    enum {
>      ...
>      FOO,
>    #define FOO FOO
>      ...
>    };

Unfortunately that trick does not work on some K&R compilers.  Not
only do they have problems with enum, they also have recursion
problems with #define FOO FOO.

This raises another issue, though.  Some pedantic compilers complain
about mixing enum and int.  So if you make the above change, the
pedantic compilers will make you want to go through bison.simple and
manually replace 'int' with 'enum yytokentype' where that is needed.
But that's not a good idea, as it would introduce compatibility
problems, since the enum type is not type-compatible with int, so
programs that contain declarations like `int yylex ();' will break.

So, all in all I would do something like the following instead.
This is a more conservative change, and is less likely to break
things.

   #ifndef YYTOKENTYPE
   # if defined (__STDC__) || defined (__cplusplus)
      /* Put the tokens into the symbol table, so that GDB and other debuggers
         know about them.  */
      enum yytokentype {
        FOO = 256,
        BAR,
        ...
      };
      /* POSIX requires `int' for tokens in interfaces.  */
   #  define YYTOKENTYPE int
   # endif
   #endif
   #define FOO 256
   #define BAR 257
   ...

This should satisfy the pedantic compilers.  This will break code that
uses non-macro FOO for its own purposes, but I think that's quite
rare; and users for which this is a problem can '#define YYTOKENTYPE
int' to work around it.

The __cplusplus is for backward compatibility with folks who compile C
programs using C++ compilers; there seem to be a good number of these
people so I wouldn't shut them off right away even if we have a
different and better way to do it now.