Toward a reentrant/thread safe parser and lexer

[John W. Eaton]

I've modified Octave's parser so that multiple parser objects may be
active at the same time if some care is used.  It is now possible to
create a parser object and call it while another parser object is
active.  This situation might happen when parsing a callback function
while the main interactive parser is waiting for input, for example.

Note that the parser and lexer are still NOT properly reentrant
because both still use global variables.  You must take care to
properly save and restore (typically with an unwind_protect object)
relevant global values before and after the nested call.

Also note that even if the global state is properly saved and restored
before and after running the parser, the parser object is not thread
safe because global variables are used and modified while the parser
is executing.  So it is not yet possible for two parsers to execute at
the same time in separate threads.  All that is currently possible is
to suspend execution of one, set up and make a nested call, then
restore the global state and return to the original parse.

First, I used "%option reentrant" for Flex and the "%api.pure"
option for Bison and created two classes, octave_lexer and
octave_parser that now contain most of the parser and lexer state that
was previously stored in global variables.  More work is needed to
move additional global variables that are used in the parser to these
new classes, but for now they are saved and restored by unwind_protect
objects and that works well enough for our needs.

As time permits, I plan to move more of the global state to the
octave_lexer and octave_parser classes.  Most of the rest of the
changes involve the functions that read input from the user, a file,
or an eval string.

BTW, while making these changes, I noticed that Octave was already making
nested calls to the parser.  The nested calls could happen when Octave
processed callback functions through the readline event hook
mechanism.  I'm not sure how this ever worked properly.  I suppose it
worked in most cases because the callback would occur when Octave was
waiting for input at the beginning of a parse, so the internal state
of the parser was the same before and after the nested parse and
things just happened to work out.  I guess we are lucky that it does
not fail often...

jwe