Re: Non-greedy wildcard possible? (Long)

[Laurence Finston]

On Mon, 17 May 2004, Magnus Lie Hetland wrote:

> Laurence Finston <address@hidden>:
> >
> [snip]
> > > Certainly. My hope was to solve that with some form of dynamic
> > > precedence -- but I can't find any reasonable way of doing that with
> > > Bison.
> > >
> >
> > I don't think "dynamic" is a suitable term with reference to the
> > grammar rules.
>
> You should take that up with the documentation authors, then <wink>
>

> The precedence used in GLR parsing, set by %dprec, is referred to as
> "dynamic precedence" (hence the name) in the documentation.
>

Point taken.

> > It is possible for `yylex()' to determine dynamically what tokens to
> > return to `yyparse()', though.
>
> Yes. That is a different matter, but also useful.
>

This is how I would go about trying to solve the problem.  I would
try
to write rules for all the kinds of markup anyone would want,
but to keep the structure of the grammar as simple and unambiguous as
possible.  I would then try to handle as much of the complexity as
possible in the scanner.  I don't find it easy to get Bison to jump
through hoops.
What I have doubts about is whether it's
really possible to formulate formal rules for the kind of very sparse
markup you want, hence my remark about "mind-reading".  I can't judge how
you've implemented this functionality in your hand-tailored version.

>
> Sure -- the hope was to use dynamic precedence to sort that out. That
> would be much easier than writing a grammar that was unambiguous
> (without any use of precedence) to begin with. This point is even made
> by the Bison documentation.
>

Others may disagree, but I think using dynamic precedence is a kludge
and I only use it as a last resort.  I certainly wouldn't want to make
it a fundamental technique of my parser.  Every time I add a rule
using addition or subtraction
it adds several shift/reduce conflicts, currently about 120.  They're
harmless, because Bison's default behavior produces the correct
results, but I'd rather not have them.  I'm pretty sure that using
different tokens for unary and binary `+' and `-' would get rid of
the problem, but I believe that the disadvantage of using an
unfamiliar notation would outweigh the advantage of not being annoyed by
the warnings.

> > In what sense is it "last"? Because only whitespace follows it up to
> > the next newline?  If `yyparse()' is to recognize it as different
> > from the other two, then `yylex()' must return a different token to
> > represent it.
>
> I'm not sure how many times it's interesting to keep repeating this,
> but still: No, yylex does not have to return a different token to
> represent it. That's the whole point. It has to be decided by a
> context-free grammar. That's why I'm asking about this. If the lexer
> could decide this sort of thing for me, there would be no problem.
>
> It seems we're just going in circles here :)
>

I agree that this is the whole point.  However, I believe that
it's a matter for the scanner and not the parser.  I wouldn't approach
the problem this way, but I hope you can get it to work.

I had a problem that reminds me of this.  Sometimes, one would like to
"get out of" a rule in `yyparse()' and "get into" another rule.  I
believe this to be impossible, except with `gotos', which is risky.
If one does it, then one must be very careful about using the semantic
values of the tokens of all the rules involved.  I _don't_ recommend
it.

The way I solved the problem was by devising a way of letting
`yyparse()' communicate with `yylex()' via the `void*' parameter.
This may or may not be useful to you.

Laurence