Re: reentrant flex, bison glue

[Akim Demaille]

>>>>> "John" == John W Millaway <address@hidden> writes:

John> Hi,

Hi!  Thanks for the details!

John> The upcoming release of flex contains some new options, two of
John> which are:

John> %option reentrant (--reentrant or -R from command line) %option
John> reentrant-bison (--reentran-bison or -Rb )

John> The 'reentrant' option generates a reentrant C scanner. All the
John> guts of flex, formerly global, are packed into a struct, of type
John> 'yyscan_t'. The upshot is that the user can declare several
John> scanners of the same type (yyscan_t*), within threaded
John> applications, or recursively, without race conditions. Read more
John> in the manual, of course.

This is really great news!  It will also help us free the memory Flex'
buffers are taking.


John> %option 'reentrant-bison' causes flex to do three things:

John> 1. Flex adds pointers of type YYSTYPE and (optionally) YYLTYPE
John> to the scanner guts. The values are expected to be passed into
John> yylex(). Flex expects these types to be defined elsewhere,
John> (e.g., y.tab.h) and NEVER touches their values.

I don't understand this.  If Bison or Yacc is used on the top of Flex,
then the .h file is to be included, which defines these guys.  Just as
you said.  So I don't understand this bit, not its relationship with
Bison.

John> 2. The macros yylval and yylloc are defined, as are the accessor
John> functions yyget_lval(), yyset_lval(), yyget_lloc(),
John> yyset_lloc(). 

Again, I don't see why these guys are there.

John> The macros can be used from within flex actions, as if they were
John> global variables. The accessor functions allow access from
John> anywhere (e.g., from within the parser).  The idea is that
John> scanner action can do something like this:

John> %% /* access yylloc as if it were global. */ \r?\n {
John> yylloc->my_line_count++; return EOL;}

Correct, but in what way would these macros would help?

John> 3. yylex() is defined with the additional arguments YYSTYPE* and
John> (optionally) YYLTYPE*.

So, if I understand correctly, this is meant to replace the
user-definition of YY_DECL?  How to you plan to take the case of the
parser passing additional arguments into account?  I'm referring to

#define YY_DECL yylex (...., my_struct *str)

?

John> The whole idea here is that a flex scanner will just "work" with
John> a bison pure parser, without much fuss for the user.

The sole difference I can see is that YY_DECL will be defined
automatically, right?


John> I wrote the bison glue code several months ago, before the
John> recent bison release. I understand that there have been some
John> changes in the pure parser. The only questions to which flex
John> needs positive answers are these:

John> Is the data type YYSTYPE defined in the header?  

Sure.

John> Is the macro YYLTYPE defined, and is it a data type? (This is
John> how flex knows to include yylloc support.)  

Nope.  If the user uses something else, it is her responsibility to
inform the scanner.  Bison does not know.

John> Is the calling convention to yylex() still the same (as in the
John> previous release of bison)?

Sure!