bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes

[Juri Linkov]

>> forward-sexp moves over a balanced parenthetical group like
>> forward-list does.  Plus forward-sexp also moves over an atom
>> such as a symbol, a number.
>> 
>> The problem is that treesit adds too much structural information
>> to such simple things as a symbol and a number.  For example, in js
>> a simple keyword "export" gets the "(export_statement export" subtree,
>> Another keyword "const" gets "(lexical_declaration kind: const", etc.
>> 
>> Therefore for such symbols forward-sexp needs to bypass the structure
>> and use simpler syntactic information to move over them like on a flat list.
>
> If you mean we should ignore the information provided by tree-sitter
> and instead use our own syntactic information, then that sounds wrong
> to me, FWIW.  Why cannot we understand enough of the tree-sitter
> structural information to move like we want?  Presumably, the
> structural information provided by tree-sitter is a portion of a parse
> tree, which to me means we should be able to move between the parse
> tree's nodes as long as we understand the tree and can interpret it in
> our terms.
>
> Aren't there some grammar-agnostic traits of tree-sitter nodes that
> would allow us to interpret the nodes in language-independent terms?
> If that is not available, then each major mode will have to provide
> treesit.el with a way to interpret the tree-sitter nodes of the
> corresponding grammar in a way that will allow sexp movement, thus
> providing an abstraction layer that treesit.el could use for the
> movement commands.

Maybe it would be possible to use something like 'flatten-tree'
on the treesit's syntax tree?  But this will require the addition
of a lot of rules to specify what nodes should be flattened.