reentrant flex, bison glue

[John W. Millaway]

--- Akim Demaille <address@hidden> wrote:
> >>>>> "W" == W L Estes <address@hidden> writes:
> 
> W> *** Two new options %option reentrant (-R) and %option
> W> reentrant-bison (-Rb).
> 
> Could you give some information about this issue on Bison mailing
> lists?  That might help the Bison team.
> 
> W> ** Support for bison variables yylval and yylloc added.
> 
> Could you also detail this point?  These variables have always been
> under the responsibility of the caller, not of Flex itself.  What
> feature is it exactly?

Hi,

The upcoming release of flex contains some new options, two of
which are:

%option reentrant (--reentrant or -R from command line) 
%option reentrant-bison (--reentran-bison or -Rb )

The 'reentrant' option generates a reentrant C scanner. All
the guts of flex, formerly global, are packed into a struct,
of type 'yyscan_t'. The upshot is that the user can declare
several scanners of the same type (yyscan_t*), within threaded
applications, or recursively, without race conditions. Read
more in the manual, of course.

The reentrant-bison option generates a scanner that is meant
to be called from a bison pure parser. The scanner is the
same -- it is the reentrant C scanner -- but with additional
support for the bison calling convention and variables. It
simply provides some convenience "glue" to bison. 

-------------------------

%option 'reentrant-bison' causes flex to do three things:

1. Flex adds pointers of type YYSTYPE and (optionally) YYLTYPE to
the scanner guts. The values are expected to be passed into
yylex(). Flex expects these types to be defined elsewhere,
(e.g., y.tab.h) and NEVER touches their values.

2. The macros yylval and yylloc are defined, as are the
accessor functions yyget_lval(), yyset_lval(), yyget_lloc(),
yyset_lloc(). The macros can be used from within flex
actions, as if they were global variables. The accessor
functions allow access from anywhere (e.g., from within the
parser).  The idea is that scanner action can do
something like this:

%%
       /* access yylloc as if it were global. */
\r?\n  { yylloc->my_line_count++; return EOL;}

3. yylex() is defined with the additional arguments YYSTYPE*
and (optionally) YYLTYPE*. 

-----------------------------------
The whole idea here is that a flex scanner will just "work"
with a bison pure parser, without much fuss for the user.

I wrote the bison glue code several months ago, before the recent
bison release. I understand that there have been some
changes in the pure parser. The only questions to which flex needs
positive answers are these:

Is the data type YYSTYPE defined in the header?
Is the macro YYLTYPE defined, and is it a data type? (This is how flex knows to
include yylloc support.)
Is the calling convention to yylex() still the same (as in
the previous release of bison)?

-John


__________________________________________________
Do You Yahoo!?
Try FREE Yahoo! Mail - the world's greatest free email!
http://mail.yahoo.com/