Re: asking for advice for changing the cfengine.el progmode to support CFEngine 3.x

[Stefan Monnier]

> SMIE is very new so I am concerned that relying on it will block Emacs
> users that haven't upgraded.

It's in Emacs-23.3.

> Can it be distributed with my library?

It's Free Software, so of course you can.

> It's also fairly hard to specify an operator precedence grammar for
> CFEngine using SMIE.

OPG are very limited, indeed, so they're difficult to use for most
languages.

> I tried it and ended up uncertain how to debug the parse tree of the
> current buffer or where my grammar's error lay.

The patch below will at least give you better feedback for the immediate
problem in your grammar.

> See below.  Perhaps I'm just too accustomed to LALR parsers, but they
> seem easier and less ambiguous.

Very likely.  With SMIE you shouldn't try to parse the language in an
exact manner, but think about parsing the minimum needed to indent
properly ;-)

Here are some concrete comments that I hope will help you.

> (defconst cfengine3-smie-bnf-table
>   '((classname)
>     (class_condition (class_condition "&&" class_condition)
>                      (class_condition "||" class_condition)
>                      (class_condition "." class_condition)
>                      ("!" class_condition)
>                      (classname))
>     (bundle_parameters (bundle_parameters "," bundle_parameters)
>                        (function_parameter))

Looks fine.

>     (bundle_name (bundle)
>                  (bundle bundle_parameters))

The second line above is outside the scope of OPG: you can't have
adjacent non-terminals in your grammar.

>     (bundle_line ("bundle" bundle_type bundle_name))

Same here.
Looking at your example, I would recommend you treat "bundle" as
an associative separator, as in

      (bundles (bundles "bundle" bundles))

This will surely strike you as really wrong, but OPG don't handle
"terminator" tokens or "start" tokens nearly as well as
separator tokens.
      
>     (promises (promise ";" promise) (promise))
>     (bundle_section (section ":" "\n" promises)
>                     (class_condition "::" "\n" promises))

Adjacent terminals are allowed in OPG, but they tend to work somewhat
poorly, so I'd recommend to try and avoid them.

>     (bundle_body (bundle_section) (bundle_section "\n" bundle_section))

I'm surprised to see \n here (and in the previous rule).  Are newlines
really significant in cfengine's syntax?

>     (bundle_block (bundle_line "{" bundle_body "}"))))

Better leave {...} as an expression of its own, so that { is an opener
and } is a closer.  I.e. let "bundle" be followed by a sequence of
<things>.
That means that

   bundle edit_line block_append_if_no_line(data)
   {
    insert_lines:
      any::
        "$(data)" insert_type => "preserve_block";
   }

will be parsed as "bundle" followed by 4 <things> (one of them being
"(data)"), which is structurally incorrect (IIUC "(data)" is actually
structurally a child of "block_append_if_no_line" rather than of
"bundle"), but we can hope that it won't matter for indentation.

> (defvar cfengine3-smie-grammar
>   (smie-prec2->grammar
>    (smie-merge-prec2s
>     (smie-bnf->prec2 cfengine3-smie-bnf-table
>                      '((assoc "\n" ";")))
>     (smie-precs->prec2 cfengine3-operator-table))))

You actually want to pass cfengine3-operator-table to smie-bnf->prec2
(as second parameter) because it's needed to disambiguate the
BNF grammar.


        Stefan


=== modified file 'lisp/emacs-lisp/smie.el'
--- lisp/emacs-lisp/smie.el     2011-06-17 01:46:30 +0000
+++ lisp/emacs-lisp/smie.el     2011-06-21 14:35:41 +0000
@@ -265,14 +265,18 @@
               ;; the trouble, and it lets the writer of the BNF
               ;; be a bit more sloppy by skipping uninteresting base
               ;; cases which are terminals but not OPs.
-              (assert (not (member (cadr rhs) nts)))
+              (when (member (cadr rhs) nts)
+                (error "Adjacent non-terminals: %s %s"
+                       (car rhs) (cadr rhs)))
               (pushnew (cadr rhs) first-ops)))
           (let ((shr (reverse rhs)))
             (if (not (member (car shr) nts))
                 (pushnew (car shr) last-ops)
               (pushnew (car shr) last-nts)
               (when (consp (cdr shr))
-                (assert (not (member (cadr shr) nts)))
+                (when (member (cadr rhs) nts)
+                  (error "Adjacent non-terminals: %s %s"
+                         (cadr rhs) (car rhs)))
                 (pushnew (cadr shr) last-ops)))))
         (push (cons nt first-ops) first-ops-table)
         (push (cons nt last-ops) last-ops-table)