Lexical analysis on buffers greater then 4GB's in size...

[Andre Stechert]

Hi there,

        I'm tokenizing some largish chunks of data with flex (bigger than
4GBs per chunk).
Because yy_n_chars (skel.c, scan.c, initscan.c) is defined as an int (32
bits by default for
64 bit targets in gcc), it fails, exiting with "fatal flex scanner internal
error--end of buffer missed".
I can submit a patch against 2.5.4 (just make the int a long int...) if
anyone thinks this is
worth fixing.  Looking forward to your reply...

Cheers,
Andre

------------------------------------------
Looksmart
Andre David Stechert
Principal Architect, Web Search
address@hidden <mailto:address@hidden>
625 Second Street
San Francisco, CA 94107
tel: (415) 348-7112
fax: (415) 348-7033
------------------------------------------