Re: [Help-bash] Posix: 2.3 Token Recognition & 2.10 Shell Grammar

[Michael Convey]

On Tue, Jul 14, 2015 at 6:01 AM, Eric Blake <address@hidden> wrote:

>
> Not that I'm aware of.  But I can at least give a layman's shot at
> trying to explain the intent:
>
> The shell allows:
>
> case in in in ) echo yes;; esac
>
> which means that the tokenizer cannot blindly treat 'in' as a keyword
> everywhere, but only in the places where the keyword is expected (the
> third token after seeing 'case' as the first token).  So, reading the
> grammar, we see (among others):
>
>
> case_clause      : Case WORD linebreak in linebreak case_list    Esac
>
> in               : In                       /* Apply rule 6 */
>
> %token  In
> /*      'in'   */
>
> 6. [Third word of for and case]
>
>    a. [ case only]
>
>     When the TOKEN is exactly the reserved word in, the token identifier
> for in shall result. Otherwise, the token WORD shall be returned.
>
>
> So the parser has seen 'case' as Case, the first 'in' as WORD, and is
> trying to determine whether the second 'in' fits the rules for
> "case_clause".  Initially, 'in' is classified as TOKEN, and we are at
> the rule for the "in" production, which says to use rule 6 to
> disambiguate the token.  Rule 6 says that the string "in" is recognized
> as a reserved word at this point of context, so the tokenizer
> reclassifies from TOKEN to In, and the grammar then accepts the clause
> as a valid sequence of tokens.  If you do anything else, like:
>
> case in \in in ) echo yes;; esac
>
> you'll get "bash: syntax error near unexpected token `\in'".  Or,
> applying the same analysis as above, the "in" production applies Rule 6
> to the TOKEN of '\in', but since it is not the literal string 'in', it
> is not recognized as a reserved word, and is not reclassified, and
> therefore the "case_clause" rule is not satisfied and you have a syntax
> error.


Eric, very helpful, thank you. I'm working on a write-up that summarizes
the token recognition and shell grammar sections at a slightly higher level
and in more understandable terms than is provided in the POSIX standard
​. Your specific example has helped me better understand the interpretation
and syntax of shell grammar.