Re: bison for nlp

[Akim Demaille]

Hi!

> Le 7 nov. 2018 à 10:09, r0ller <address@hidden> a écrit :
> 
> Hi Akim,
> 
> The file hi_nongen.y is just left there as the last version that I wrote 
> manually:) If you check out any other hi.y files in the platform specific 
> directories (e.g. the one for the online demo is 
> https://github.com/r0ller/alice/blob/master/hi_js/hi.y but you can have a 
> look in hi_android or hi_desktop as well) you’ll see how they look like 
> nowadays.

You have tons of

   logger::singleton()==NULL?(void)0:logger::singleton()->log(2,"vm is NULL!");

you could introduce logger::log, or whatever free function,
that does that for you instead of having to deal with that
in every call site.

> Numbering tokens was introduced in the very beginning and has been questioned 
> by myself quite a many times if it's still needed. I didn’t give a hard try 
> to get rid of it mainly due to one reason: I want to have an error handling 
> that tells in case of an error which symbols could be accepted instead of the 
> erroneous one just as bison itself does it but in a structured way (as bison 
> returns that info in an error message string).

Where are these numbers used?

> Though, I could not come up with any better idea when it comes to remapping a 
> token to a symbol. As far as I know bison uses internally the tokens and not 
> the symbols for the terminals and it's not possible to get back a symbol 
> belonging to a certain token. That's it roughly but I'd be glad to get rid of 
> it. However, if it's not possible and poses no problems then I can live with 
> it. By the way, are there any number ranges or specific numbers that are 
> reserved?

Some numbers are reserved, yes: 0 for eof and 256 for error (per POSIX).  For 
error, Bison can accommodate if you use 256.  EOF must be 0.


> Not using the C++ features of bison has historical reasons: I started writing 
> the project in C and even back then I used yacc which I later replaced with 
> bison. When I started to shift the project to C++ I was glad that it still 
> worked with the generated C parser and since then I never had time to make 
> such an excursion but it'd be great. I also must admit that I wasn't really 
> aware of it. The only thing I read somewhere was that bison has a C++ wrapper 
> but have never taken any steps into that direction.

I don’t know what you mean here: this is bison itself, there’s
no need for a wrapper, and the deterministic parser itself is
genuine C++, not C++ wrapping C.  The GLR parser in C++ though _is_
a wrapper for the C GLR parser.

> Now I think I'll find some time for it -at least to check it out:) Could you 
> give me any links pointing to any tutorial or something like that? It’d be 
> very kind if you could help me in taking the first steps, thanks!

I would very like to have your opinion on the open section of the
documentation about C++.  It’s recent, and it probably needs polishing.

https://www.gnu.org/software/bison/manual/bison.html#A-Simple-C_002b_002b-Example