Re: Making statement delimiter optional ...

[Hans Aberg]

At 18:48 +0100 2003/06/02, Ricardo Rafael wrote:
>       I have managed to implement a script reentrant compiler with Flex
>and Bison. The language grammar is very similar to JavaScript (I could
>say it is almost 100% ECMA script compatible). I have used ';' as a
>statement delimiter.
>
>    Are there any hints on the following: How can I quickly make the
>statement delimiter optional? I have already made some experiments but
>the amount of a) work involved (in rewriting the grammar) or b) the
>shift/reduce and reduce/reduce conflicts introduced are not worth the
>effort!
>
>    I also have thought removing statement delimiter, but again, I
>introduced a lot of bison conflicts!
>
>    Is there anyone who has had the same experience? Any help greatly
>appreciated!

One can take as example the Bison grammar itself (used to parse the .y
files)! Then the ";" already were optional, but it turns out that it made
the grammar to not even be LALR(1). Several tweaks were suggested. One from
me was to tweak the lexer so that it could scan forth and then insert
"virtual" terminators ";".

Another tweak is mentioned in the Bison manual, to use GLR parser. It gives
an example for the C++ grammar. The reason it is convenient here, though is
that it is not possible to determine which parse to use until runtime.

Another example is from a language of my own that I am working on, for
metamathematical inference (proof verification). When I recently tried to
remove a ":" delimiter for quantifiers, I ended up with a few shift/reduce
conflict that proved very difficult to remove. Eventually I decided to put
the ":" back.

My own conclusion is that the statement delimiters help the parsing, making
the implementation much easier. Probably it makes error detection easier
too, as the parse can make shorter lookaheads.

  Hans Aberg