Re: flex snapshot/beta

[John W. Millaway]

> If you have not already made it that way, I think one should be able to
> choose if one wants to have EOL to be \n (UNIX), \r (MacOS), \r\n (DOS,
> MSOS) or a combination of them (like all three). If the line counting
> feature is on and \r\n is admitted as EOL, it should only increment line
> numbers by one, not two.

The change mentioned is only for flex's INTERNAL scanner (not the generated
scanner's input.) It only affect EOLs in "foo.l". The EOL test used in the
GENERATED scanner is unchanged, and is merely a check for '\n'.

But you're right: If we're going to have the optional automatic EOL test,
(given the performance penalty, yadda yadda yadda) then we should allow the
test to be configurable. At least, one should be able to specify one of the
three common cases you listed above.

The only tricky case is when the user wants flex to detect "\r\n", but then
fouls it all up by writing patterns that split the token, like this:

  /* User tells flex that \r\n is the EOL sequence. */
%%
  /* But the grammar may explicitly want CR and LF
     as separate tokens. */
\r   { return CR; } /* No EOL here, says flex. */
\n   { return LF;}  /* No EOL here either!. */

%%

Now, I'm lying a little, because the input is buffered, so flex could
technically look behind in the buffer for the start of the eol sequence.


__________________________________________________
Do You Yahoo!?
Try FREE Yahoo! Mail - the world's greatest free email!
http://mail.yahoo.com/