[GLISS] Existing syntax abominations

[David Kastrup]

Well, the host of different syntaxes to \tempo aside, here are some
other nuisances:

Sometimes it is important to be able to parse some expression without
further lookahead, for example because lexer modes need switching.  I am
just now experimenting with code where the _lexer_ will transparently
call music functions in its own parser copy, inserting the result back
into LilyPond.  Now I get the following output for cue-clef-new-line.ly:

input/regression/cue-clef-new-line.ly:14:20: error: unknown escaped string: 
`\vI'
\addQuote vIQuote { 
                    \vI }
input/regression/cue-clef-new-line.ly:14:20: error: syntax error, unexpected 
STRING
\addQuote vIQuote { 
                    \vI }

The input is

vI = \relative c'' { \clef "treble" \repeat unfold 40 g4 }
\addQuote vIQuote { \vI }

Huh?  Why is \vI undefined at the time \addQuote is called?  Now since
\addQuote is called in the lexer in this LilyPond version, it is called
when the preceding code is asking for a lookahead token.  Why on Earth
would the preceding code ask for a lookahead token to finish that
assignment?

Calling lilypond with -ddebug-parser tells us:

Entering state 55
Reducing stack by rule 134 (line 1007):
   $1 = nterm braced_music_list (: )
-> $$ = nterm sequential_music (: )
Stack now 0 2 6 168 296
Entering state 57
Reducing stack by rule 157 (line 1093):
   $1 = nterm sequential_music (: )
-> $$ = nterm grouped_music_list (: )
Stack now 0 2 6 168 296
Entering state 61
Reducing stack by rule 155 (line 1088):
   $1 = nterm grouped_music_list (: )
-> $$ = nterm music_bare (: )
Stack now 0 2 6 168 296
Entering state 60
Reducing stack by rule 152 (line 1082):
   $1 = nterm music_bare (: )
-> $$ = nterm composite_music (: )
Stack now 0 2 6 168 296
Entering state 136
Reading a token: Starting parse
Entering state 0
Reading a token: Next token is token "(music-function-call)" (: #<Music
function

So here is where we have "composite_music", and the following \addQuote
is called prematurely in the search of a lookahead token.  Why?  Let's
look at state 136 in the parser:

state 136

  130 music_assign: composite_music .  ["end of input", error, "\\repeat", 
"\\alternative", "\\default", ':', '(', ')', '[', ']', '~', '^', '_', "--", 
"__", "\\!", EVENT_IDENTIFIER, E_UNSIGNED, "\\[", "\\]", "\\(", "\\)", "\\<", 
"\\>", DURATION_IDENTIFIER, REAL, UNSIGNED, NUMBER_IDENTIFIER, "\\accepts", 
"\\alias", "\\book", "\\bookpart", "\\change", "\\chordmode", "\\chords", 
"\\consists", "\\context", "\\defaultchild", "\\denies", "\\description", 
"\\drummode", "\\drums", "\\figuremode", "\\figures", "\\header", 
"\\version-error", "\\layout", "\\lyricmode", "\\lyrics", "\\lyricsto", 
"\\markup", "\\markuplist", "\\midi", "\\name", "\\notemode", "\\override", 
"\\paper", "\\remove", "\\revert", "\\score", "\\sequential", "\\set", 
"\\simultaneous", "\\tempo", "\\type", "\\unset", "\\with", "\\new", "<", "<<", 
">>", "\\", "\\~", FIGURE_OPEN, LYRIC_MARKUP, MULTI_MEASURE_REST, 
"(backed-up?)", "(reparsed?)", CHORD_REPETITION, CONTEXT_MOD_IDENTIFIER, 
DRUM_PITCH, PITCH_IDENTIFIER, FRACTION, LYRICS_STRING, LYRIC_MARKUP_IDENTIFIER, 
MARKUP_IDENTIFIER, MARKUPLIST_IDENTIFIER, MUSIC_IDENTIFIER, NOTENAME_PITCH, 
RESTNAME, SCM_IDENTIFIER, SCM_TOKEN, STRING, STRING_IDENTIFIER, 
TONICNAME_PITCH, '-', '{', '}', '|']
  227 new_lyrics: . "\\addlyrics" address@hidden composite_music
  229           | . new_lyrics "\\addlyrics" address@hidden composite_music
  230 re_rhythmed_music: composite_music . new_lyrics

    "\\addlyrics"  shift, and go to state 202

    $default  reduce using rule 130 (music_assign)

    new_lyrics  go to state 203

    Conflict between rule 130 and token "\\addlyrics" resolved as shift 
(COMPOSITE < "\\addlyrics").


Look and behold: after the closing brace of the sequential music, the
expression is not finished because LilyPond has to see whether there is
an \addlyrics after that, as it would become part of the expression.

Well, it seems my "stealthy" music function call in the lexer can't work
just as stealthily as that since a brace-enclosed music expression is
potentially incomplete.  That's actually rather bad news for other
potentially mode-switching commands as well.

So \addlyrics is on my blacklist of unnecessarily surprising constructs.

\addlyrics is actually something calling itself "re_rhythmed_music" in
the parser.  That concept would make independent sense, but it is only
available for lyrics, not anything else.

Another parser baddy is \alternatives since it means that no \repeat
expression can be considered complete without checking for potentially
following alternatives.  It would make more sense if the alternatives
were written _inside_ of the repeat.  Much more sense actually.

Then we have \override Grob #'this #'that = 7 which needs to get
reverted with \revert Grob #'(this that) rather counter-intuitively, to
the degree that people complain about a recently introduced warning
resulting from \revert Grob #'this #'that (which never did what people
thought it would, ignoring all but the first Scheme expression).

Then strings in lyrics are sufficiently differently delimited from the
way strings in markups are.  For example, they can contain unquoted
curly braces in some positions.  In my opinion, lyrics (which can be
interspersed with durations) have even less business to allow curly
braces as part of words without using quote marks than markups have.

-- 
David Kastrup