Re: Using bison without Flex

[Hans Aberg]

> On 24 Apr 2015, at 15:47, John R. Levine <address@hidden> wrote:

>> Then, when the lexer finds it cannot scan forward anymore, by a character or 
>> EOS, it takes the longest match according to some rules and puts the rest 
>> stuff back into the stream for rescanning.
> 
> That's not how flex works, unless you use the uncommon '/' trailing context 
> operator.  It turns all of the patterns into a DFA and uses that to match the 
> input, not normally looking more than one character ahead. The DFA that bison 
> creates to do shifting should be able to do anything that the flex DFA can do.

The DFA can only tell whether a full string is a match or not. When scanning 
forwards for the longest match, one must continue until there is a mismatch 
while recording the position for the final states one finds. The match is the 
final state with the highest position number: what remains must be put back.