Re: bison for nlp

[Akim Demaille]

Hi!

> Le 6 nov. 2018 à 17:32, r0ller <address@hidden> a écrit :
> 
> To whoever it may concern,
> 
> I’m using bison to create an open source nlp tool and it's kind of useable 
> already (see online demo at https://r0ller.github.io/alice).

Wow!  That’s impressive.

> The bison source itself is generated from the grammar rules (which can at 
> most be binary branching) that are stored in an sqlite db file along with all 
> other specs of the language being modeled.

So a file like hi_nongen.y is generated, or not?  Why do you number your tokens?

%token  t_Con 1
%token  t_ENG_A 2
%token  t_ENG_Adv 3
%token  t_ENG_Det 4
%token  t_ENG_N_stem 5
%token  t_ENG_N_lfea_Pl 6

?

Are there features missing in Bison could help?

I see that this file is C++, but you’re not using the C++ features of Bison, is 
that on purpose?  Would you want some assistance to move to C++?

I also see you use std::endl.  Really, you shouldn’t: stick to ‘\n’.  It’s 
nicer to read, and faster (you are unlike to need to flush each line).  There’s 
an urban legend in the C++ world that std::endl would be portable on Windows 
(i.e., generates \r\n), but actually it’s already the case for \n.

https://gitlab.lrde.epita.fr/vcsn/vcsn/commit/733163ad406125460e8deffb6508b16f570b9bff
https://www.youtube.com/watch?v=lHGR_kH0PNA

> The rules of course cover the grammar while the actions carry out semantic 
> checks. The phonological and morphological analyses are covered by foma 
> (https://fomafst.github.io) which also uses bison.

https://github.com/mhulden/foma/blob/4e98d61ce67babe1555e1e1b04d5a65bc3b5a03a/foma/regex.y#L205
%expect 686

OMG!  And it’s not GLR, this is plain old LALR(1).  How can one trust such a 
number of conflicts?  If this is really means, then that’s probably a place 
where a feature such as 
https://lists.gnu.org/archive/html/bison-patches/2013-02/msg00105.html would 
shine.


> As I mentioned the whole stuff is open source and free so please, don't take 
> this as an ad. I just wanted to say thank you to all who develops/maintains 
> bison by showing what you made possible.

That’s very kind, thanks a lot!