[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Bison 1.875 NOT supporting Flex 2.5.27 reentrant C scanners
From: |
Jeannot Langlois |
Subject: |
Bison 1.875 NOT supporting Flex 2.5.27 reentrant C scanners |
Date: |
Sat, 15 Mar 2003 21:44:04 -0500 |
Hello,
I am actually trying to generate a REENTRANT parser with Bison 1.875 (in C)
using a REENTRANT scanner (also in C) generated by Flex 2.5.27. I need
everything to be in C. The parser will be used in a multithreaded application
(which is also in C) and will exist in multiple "instances", as different
threads will be calling "yyparse" simultaneously.
Unfortunately, it seems to me (from all the posts and informations I've read so
far -- please feel free to correct me if I am wrong) that parsers generated by
Bison 1.875 are NOT calling REENTRANT C Flex scanners properly... at least yet.
The problem is caused by the YYLEX definition NOT providing the "yyscanner"
opaque structure parameter which contains the Flex scanner's state in the
"yylex" function call occuring within "yyparse".
I've found a simple way to fix this problem by modifying the parser C source
file AFTER it has been generated by Bison so the call to "yylex" is done
properly by "yyparse" when the Bison parser C source file is generated with the
proper symbol (currently YYREENTRANT_FLEX_PATCH).
This temporary solution isn't very practical because all modications are lost
when regenerating the parser. I thought Bison developers would be interested
in fixing the problem, so I am writing this little text detailing how to
proceed with the temporary fix in the Bison-generated parser C source file.
I did not send this email to the Bison Patches mailing list because:
- this is not a problem which is specific to a grammar,
- I am not very familiar with the Bison sources,
- this patch is for a released version of Bison (1.875),
- this function call problem is rather general,
- the file to patch (the parser C source file) varies with each different
parser generated by Bison.
So I guess submitting a patch in this case would not be very useful.
I personnally think it would be a great idea to add support for REENTRANT Flex
scanners in Bison by adding a command-line switch like
"--reentrant-C-flex-scanner" which would activate the patch detailed below (the
user knows wether he is using a REENTRANT C Flex scanner when he generates the
Bison parser).
So here's the patch...
--------------------------------------------------------------------------------------------------------------------------------
The following procedure explains how to modify an existing C parser (generated
with Bison 1.875) to support a reentrant C Flex 2.5.27 scanner:
1. [parser C source file]:
After the YYSTYPE union declaration (more precisely after the "/*Copy
the second part of user declarations.*/" comment),
add the following:
#ifdef YYREENTRANT_FLEX_PATCH
#include "scanner.h"
#endif
("scanner.h" is a header file containing the reentrant C API of the
FLEX 2.5.27 scanner and was generated with
the --header-file="scanner.h" parameter when the REENTRANT C FLEX
scanner was generated. I know some people are
discussing that including this API in a Bison parser is not a good
idea, but it seemed to me that I had no other choice).
2. [parser C source file]:
Modify the YYLEX definition so it looks like this:
#ifdef YYREENTRANT_FLEX_PATCH
/*new YYLEX definition which supports REENTRANT flex scanners*/
#ifdef YYLEX_PARAM
#define YYLEX yylex(&yylval, YYLEX_PARAM, yyscanner)
#else
#define YYLEX yylex(&yylval, yyscanner)
#endif
#else
/*original YYLEX definition which works fine with NON-REENTRANT
flex scanners*/
#ifdef YYLEX_PARAM
#define YYLEX yylex(&yylval, YYLEX_PARAM)
#else
#define YYLEX yylex(&yylval)
#endif
#endif /*YYREENTRANT_FLEX_PATCH*/
3. [parser C source file]:
At the beginning of the "yyparse" function, among the other variable
declarations, add the following declaration:
#ifdef YYREENTRANT_FLEX_PATCH
yyscan_t yyscanner;
#endif
This declares an opaque scanner state object.
4. [parser C source file]:
Just before the call to YYDPRINTF((stderr, "Starting parse")) in the
"yyparse" function, add the following:
#ifdef YYREENTRANT_FLEX_PATCH
yylex_init(&yyscanner);
#endif
This initializes the opaque scanner state object declared previously.
5. [parser C source file]:
Just before the "return" instruction at the end of the "yyparse"
function, add the following:
#ifdef YYREENTRANT_FLEX_PATCH
yylex_destroy(yyscanner);
#endif
This destroys the opaque scanner state object declared previously.
6. [probably in a Makefile or at compile time]:
Define YYREENTRANT_FLEX_PATCH=1 (-DYYREENTRANT_FLEX_PATCH=1) when
compiling the parser C source file with GCC.
This will enable support to call a reentrant FLEX scanner, as explained
in the previous steps.
I guess this YYREENTRANT_FLEX_PATCH definition should occur when the
user selects the "--reentrant-C-flex-scanner"
parameter when generating a Bison parser, as I've suggested above... (?)
--------------------------------------------------------------------------------------------------------------------------------
So that's it...
Hope this little patch helps...
Regards,
--
Jeannot Langlois
address@hidden
--
______________________________________________
http://www.linuxmail.org/
Now with e-mail forwarding for only US$5.95/yr
Powered by Outblaze
- Bison 1.875 NOT supporting Flex 2.5.27 reentrant C scanners,
Jeannot Langlois <=