Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Jens Schmidt]

On 2023-08-27  09:43, Ihor Radchenko wrote:
> Samuel Loury <konubinix@gmail.com> writes:
> 
>> IMHO, "-tag&-todo=TODO" is totally ok. I even imagine we could say that
>> & and | are forbidden to say anything else than AND and OR and people
>> would be ok with that.
> 
> Actually, explicit & or | might be an easier way to not worry about
> escaping things. Except escaping & or | themselves.
> It might be the preferred way to put into the manual.
> 
>> IOW, I wonder of the time and effort to deal with optional & is worth
>> it. WDYT?
> 
> We cannot remove the optionality of &, because doing so will break
> existing user setups. But we can certainly change our recommendations in
> the manual.
> 
> Though pure tag matcher makes more sense with implicit &:
>  +tag1-tag2+tag3... vs +tag1&-tag2&+tag3
> Especially in the interactive agenda filters.

I feel it's a bit hard to reply to this message in the right places, so
I'll do a plain bottom post, sorry.

TL;DR:

- I think we cannot make "&" mandatory because of backward compatibility.

- Even if we made "&" mandatory, it would not really solve the quoting
  problem, since the parser is rather hacky and other quoting and
  context issues would still be there.

- But then I cannot see any real reason why we should recommend using
  "&" in the manual.

- Finally, we should hope for and work towards a "real parser", as Ihor
  has mentioned already in a previous post:

  (beginning of thread)
  
https://list.orgmode.org/orgmode/9132e58f-d89e-f7df-bbe4-43d53a2367d2@vodafonemail.de/
  (mentioning of peg)
  https://list.orgmode.org/orgmode/87wmyendr7.fsf@localhost/

Details, as far as still required:

The whole tag/property query parser seems to have a long tradition.  For
example, there is still that TODO-state special case that allows for
queries like this (example ripped from the manual):

  work/WAITING
     Same as work+TODO​="WAITING"

So people *have* been worrying about a) query brevity and b) backward
compatibility, and I think we should do so as well.

Then the current parser is horribly ad-hoc-ish and, as a plain
regexp-based parser, context insensitive.  For example, a tag regexp
match

  {regexp}

is matched simply as

  {[^}]+}

that is, you have no chance whatsoever to quote the closing curly
parenthesis.  Likewise for double-quoted strings.

In other words, even if we make | or & mandatory, there will still
remain a lot of ambiguities stemming from the rest of the parser.  And
I'm too lazy to think about how we could quote & in that whole picture.
(| is sort of handled already, but again in a completely
context-insensitive way.)

This all calls for a proper parser, based on peg or bovine or whatever.
Hopefully that parser would still keep backward compatibility, but
that's probably wishful thinking.