push parser/lexer interface

[John W. Eaton]

I've made a number of changes to the parser and lexer recently in
order to allow a push parser interface to really work.

As I described previously, I've made it possible for multiple parser
instances to be used.  See my message here:
https://mailman.cae.wisc.edu/pipermail/octave-maintainers/2013-February/032338.html)

In addition to what I described in my earlier message, I've now moved
many global variables into the octave_parser and octave_lexer
classes.  There are still more global variables to deal with, but very
few parser-related variables need to be saved/restored using
unwind_protect now.

Next, I've created a "push" interface for the parser and lexer.  It's
now possible to send a sequence of character strings to the parser and
have it tell you when a complete statement is ready to be evaluated.
This is similar to passing program statements to the eval function
except that the push parser can handle partial statements while eval
must have a complete statement.  The eval function also evaluates the
statement while the push parser only parses the input and generates a
parse tree.  It's up to the caller to execute (or do something else
with) the resulting parse tree.

With the changes I've checked in, you can write the following three
functions

  #include <octave/config.h>
  #include <defun-dld.h>
  #include <pt-stmt.h>
  #include <pt-eval.h>
  #include <parse.h>

  static octave_push_parser my_parser;
  static tree_evaluator my_evaluator;

  DEFUN_DLD (push_parse, args, ,
    "-*- texinfo -*-\n\
  @deftypefn {Built-in Function} {} push_parse (@var{input}, @var{eof})\n\
  Play with the push parser interface.\n\
  @end deftypefn")
  {
    octave_value retval;

    int nargin = args.length ();

    if (nargin == 1)
      {
        std::string input = args(0).string_value ();

        if (! error_state)
          retval = my_parser.run (input, false);
        else
          error ("expecting character string as input");
      }
    else
      print_usage ();

    return retval;
  }

  DEFUN_DLD (push_parse_eval, args, ,
    "-*- texinfo -*-\n\
  @deftypefn {Built-in Function} {} push_parse_eval ())\n\
  Play with the push parser interface.\n\
  @end deftypefn")
  {
    octave_value retval;

    if (args.length () == 0)
      {
        if (my_parser.stmt_list)
          my_parser.stmt_list->accept (my_evaluator);
      }
    else
      print_usage ();

    return retval;
  }

  function octave_in_octave ()
    prompt = ">> ";
    while (true)
      in = input (prompt, "s");
      if (! isempty (in))
        in = cstrcat (in, "\n");
        status = push_parse (in);
        if (status == 0)
          push_parse_eval ();
          prompt = ">> ";
        else
          prompt = "";
        endif
      endif
    endwhile
  endfunction

and do the following at the Octave prompt:

  octave:1> autoload ("push_parse_eval", fullfile (pwd, "push_parse.oct"))
  octave:2> octave_in_octave
  >> for i = 1:5
  i
  end
  i =  1
  i =  2
  i =  3
  i =  4
  i =  5
  >> if (i == 5)
    graphics_toolkit fltk
    sombrero ();
  end

For now, the usual pull parser is still used for all parsing jobs in
Octave and it will probably always be used for parsing files as I
can't see any real advantage to using a push parser for that job.

But now we can gather lines of input from anywhere and push them at
Octave one at a time and the parser can handle partial statements.
We aren't restricted to using Octave in a classic tty now.  It should
be possible to "embed" Octave in a GUI that has complete control over
the terminal window.  We don't have to run Octave in a separate
thread, or use a pty to communicate with it.

jwe