Re: reentrant flex, bison glue

[Akim Demaille]

>>>>> "John" == John W Millaway <address@hidden> writes:

>> You missing something: you are talking of bison producing LALR C
>> parsers only.  I'm talking about Bison.  We already have a
>> functional C++ Bison.  We should have DLR, and maybe other
>> techniques.

John> How does LALR/DLR affect the scanner API? 

I don't think it does.  Actually, I'm sure it doesn't.  My point was
not that in itself it would invalidate something, but that in itself
it frees us from some old interface, as it's no longer the same old
Yacc stuff.

John> I would LOVE to put token buffer support into flex, but that's
John> for a different day. 

I'm not sure I see the relationship here.

John> Does the new bison support a scanner object? 

Nope.  The last time it was tried here, it was disastrous: something
like a hundred times slower that the C scanner.  I never tried again.


John> The bison glue was developed before I saw any action in the
John> bison list about the new stuff. If it totally breaks the new
John> bison model, then by all means let's remove it from flex.

I don't think it breaks something, really.  I think it is too partial
an answer because the interface in itself is much more free than
merely yylloc and yylval.  Names can change.  People may pass
additional args. etc. etc.

I'm very worried when I see two packages having an increased
impedance.  Independence was the keyword up to now.  I'm worried of
having to see in a NEWS file

NEWS in Flex 3.0

- brand new lexical blah blah.
- it requires Bison 3.1 to use reentrant-bison because of the impact
  of the brand new syntactical blah blah.

NEWS in Bison 3.1

- brand new syntactical blah blah.
- it requires Flex 3.0 to use reentrant-bison because of the impact
  of the brand new lexical blah blah.

I'm just worried, nothing to argument unfortunately.  It's only in the
guts I fear.



>> Basically, I guess the reason I could not understand you is that I
>> don't expect Bison and Flex to try to be smart with my code.  I
>> want them to expand the rules, period.  I'm the chief for the rest
>> of the code.

John> I disagree entirely. 

So that's really the bottom line :)  We found it :)

John> Flex is not just a tool that builds a giant switch statement. It
John> is a scanner generator. Look at some other tools. If Antlr
John> (http://www.antlr.org/) had C support, I'd use it over
John> flex/bison without a second thought.

Agreed.  That's also why I'm trying to improve Bison.

John> Antlr is better in every way than flex/bison. The challenge is
John> to improve flex to "catch up" to these modern tools. That's what
John> you're doing with bison, isn't it?

Correct.


>> I suppose it means the user must not have code using the names
>> yylloc and yylval but as local variables?

John> No. If a user wants to declare variables "yylloc" and "yylval"
John> for some god awful reason, that's perfectly fine. It will not
John> clash with flex, since those are struct members in flex. 

I understood that you rewrote these into an access in the struct.

John> I think the user won't be able to access those variables from
John> within the scanner actions, though. But even so, the user
John> shouldn't prefix her variables with "yy" if she doesn't expect
John> possible clashes with flex/bison!

Agreed.  But that's to be documented.

>> I also guess you can't use this feature when you renamed yy to foo? 
>> (I mean, using Bison's --name-prefix, not Flex's equivalent
>> option).

John> No, look in the flex tests/ directory. The bison glue examples
John> use the prefix "test" or something like that.

How does Flex knows under what name yylval and yylloc are exported
from Bison???