Re: Standardizing tree-sitter fontification features

[Yuan Fu]

> On Nov 24, 2022, at 5:13 PM, Randy Taylor <dev@rjt.dev> wrote:
> 
> On Thursday, November 24th, 2022 at 17:16, Yuan Fu <casouri@gmail.com> wrote:
> 
>> 
>> For tree-sitter-based major modes, fontification rules are categorized into 
>> “features”, which can be individually turned on/off. I think it would be 
>> good to have a standardized list of common features and their precise 
>> meaning defined. We’ve been working on these fontification rules for some 
>> time and arrived at a reasonable baseline, and now it’s a good time to 
>> discuss and bless it, I think.
>> 
>> Right now we have:
>> 
>> Basic tokens:
>> 
>> delimiter ,.;
>> operator = != ||
>> bracket []{}()
>> 
>> constant true, false, null
>> number
>> keyword
>> comment
>> string
>> string-interpolation f"text {variable}"
>> escape-sequence "\n\t\\"
>> function every function identifier
>> variable every variable identifier
>> type every type identifier
>> property a.b <--- highlight b
>> key { a: b, c: d } <--- highlight a, c
>> error highlight parse error
>> 
>> More abstract ones:
>> 
>> assignment: the LHS of an assignment (thing being assigned to), eg:
>> 
>> a = b <--- highlight a
>> a.b = c <--- highlight b
>> a[1] = d <--- highlight a
>> 
>> definition: the thing being defined, eg:
>> 
>> int a(int b) { <--- highlight a
>> return 0
>> }
>> 
>> int a; <-- highlight a
>> 
>> struct a { <--- highlight a
>> int b; <--- highlight b
>> }
>> 
>> There are also language-specific features, but they are not the focus here.
>> 
>> Once we agree on a list of standard features and their definition, the next 
>> step would be to figure out how should a major mode introduce its supported 
>> features to a user (major mode docstring + link to manual for standard 
>> features?).
>> 
>> Also, some of the features are very busy, it would be good if we can disable 
>> they by default. The default value of font-lock-maximum-decoration is t, 
>> meaning use everything, which is not very helpful...
>> 
>> Yuan
> 
> Looks good!
> 
> key should be considered property IMO, and that's how we're highlighting 
> things now.

I agree.

> 
> I wonder if assignment and definition are really worth having (and would 
> prefer to do without them), since they should be covered by the variable, 
> function, type and property features.

They are definitely useful. They are the things we currently highlight, and for 
a reason. Personally I only want to highlight identifiers in definition and 
assignment, not every occurrence of them. Since so much of a program consists 
of variable and function identifiers, highlighting all of them looks almost 
like highlighting everything. I just want some visual cues on the program 
structure, not programming in skittles :-)

> 
> I would also add:
> - misc-punctuation, for anything not considered a delimiter or bracket. Most 
> modes would use this for any special punctuation they've got.

Is there any examples? Maybe just merge delimiter and punctuation together?

> - (maybe) literal instead of number? That way there is a group for chars too 
> (and any other literals if there are any?). Or a char feature in addition to 
> the existing number one. I'm undecided...

Literal seems to encompass numbers, strings, chars, and constants. So I don’t 
know how does it fit. We could add char to string feature.

> 
> Maybe a slight tangent but I also suggest we alphabetize all of these; both 
> the queries and the list of features. I'll send a patch to do that myself 
> once things cool down a bit. Although anything that overrides will need to go 
> at the bottom to make sure it gets applied.

Good idea :-) A tangent of your tangent: how did you alphabetize them? Did you 
use the sort-word package on EmacsWiki?

Yuan