Re: [O] question on org-element-interpret-data and when it works

[Thorsten Jolitz]

John Kitchin <address@hidden> writes:

Hello John,

> I am trying to find some ways to programatically modify org-elements
> that use fewer regexps and motion commands. It seems like org-dp
> (https://github.com/tj64/org-dp) was intended to do that

thats right, that's it's exact use case

> but it is not clear enough how you might use it, and it also doesn't
> seem to support plain-lists yet.

it's actually split into core and lib functionality, org-dp.el being the
core and org-dp-lib.el being the lib.

AFAIK it org-dp does work, and in its core it's based on just two
commands, one for CREATE and one for REWIRE (=modify) org elements
(locally, not replacing Nicolas org element framework, but rather making
it easy to use locally without the need for a whole parse tree).

The basic idea:
 - internally, all org elements look alike (plists)
 - org elements have many properties, but we are interested only in
   those used by the org element interpreter (that creates org syntax
   out of Emacs Lisp plists). These are surprisingly few.
 - luckily, plists ignore all elements that are not accessed, making
   transformation between different plists for different org elements
   much easier

You should be able to do anything you want with org elements with just
these two functions, org-dp-create and org-dp-rewire.

There is a generic prompt function two. Never write interactive specs
again, just use this one prompt functions for all org element related
prompts (or utility commands based on the prompt workhorse).

And then there is this fast and simple mapping function for org-dp:

,----[ C-h f org-dp-map RET ]
| org-dp-map is a Lisp function in ‘org-dp.el’.
| 
| (org-dp-map FUN-WITH-ARGS RGXP &optional MATCH-POS BACKWARD-SEARCH-P
| BEG END SILENT-P)
| 
| Apply quoted FUN-WITH-ARGS at every RGXP match.
| 
| [...] 
| In contrast to other mapping functions in Org-mode, this mapping
| function does not collect any information about mapped elements,
| it simply moves point quickly to all positions in a buffer(range)
| that are matched by a (forward) regexp-search and applies one of
| ‘org-dp’’s or ‘org-dp-lib’’s functions locally at that
| point (i.e. without any context information other than that about
| the parsed element-at-point).
| 
| When calling FUN ‘org-dp-create’, or ‘org-dp-rewire’ with
| argument ELEMENT given, no parsing at all takes places, but newly
| created of modified elements can be inserted at point.
| 
| This mapping function wraps its body in ‘save-excursion’ and
| ‘save-match-data’ calls, so point position and global match-data
| are preserved. It does not widen the buffer before executing its
| body, so buffer restrictions are respected. 
`----

> What I imagined happening is that I would get the element to modify as a
> data structure, modify the data structure, and replace the old element
> with an interpreted version of the modified data structure.

Thats exactly what this central org-dp workhorse function does:

,----[ C-h f org-dp-rewire RET ]
| org-dp-rewire is a Lisp function in ‘org-dp.el’.
| 
| (org-dp-rewire ELEM-TYPE &optional CONTENTS REPLACE AFFILIATED ELEMENT
| &rest ARGS)
| 
| Rewire element-at-point or ELEMENT (if given).
| 
| [...] 
| ELEM-TYPE is one of the types in ‘org-element-all-elements’. If
| it is nil, the element type of the original element is used. ARGS
| is a plist consisting of key-val pairs of all other keyword
| arguments given, defining the (rewired) element’s properties.
| 
| The former value of an element property can be reused in the
| creation of a new value by giving a ‘lambda’ expession or
| function taking two arguments (instead of just a value) to a
| key. The first argument will then be replaced by the property’s
| former value when applying the function. The second argument
| should be the parsed element itself, enabling access to its type
| and all its properties inside of the lambda expression.
`----

But, as Nicolas said in his answer, plain lists, tables, properties
etc are nested org elements, what is most obvious with tables:
a table is just a container for table rows that hold the actual data
(with some meta data in the container).

Thats why org-dp-lib.el has several related functions:

,----
| org-dp-lib.el
| 40:(defun org-dp-wrap-in-block (&optional lines user-info &rest prompt-spec)
| 204:(defun org-dp-toggle-headers (&optional action)
| 290:(defun org-dp-org-props ()
| 307:(defun org-dp-filter-node-props (filter &optional negate-p verbose-p)
| 366:(defun org-dp-create-table (row-lst &optional tblfm table-el-p insert-p)
| 416:(defun org-dp-create-plain-list (item-lst &optional insert-p)
| 452:(defun org-dp-create-property-drawer (node-prop-lst &optional insert-p)
`----

They should hopefully be pretty well documented, since I use to do that.

To get started with org-dp, you really need these 3 (or 4)
functions. They spare you typing, and more important, they tell which
properties of an org element matter for its interpretation.

CREATE:

,----[ C-h f tempo-template-org-dp-create RET ]
| tempo-template-org-dp-create is an interactive Lisp function.
| 
| (tempo-template-org-dp-create &optional ARG)
| 
| Insert org-dp-create template.
| 
| [back]
`----

or this one with explanations to get started (not so much for productive use):

,----[ C-h f tempo-template-org-dp-create-with-comments RET ]
| tempo-template-org-dp-create-with-comments is an interactive Lisp
| function.
| 
| (tempo-template-org-dp-create-with-comments &optional ARG)
| 
| Insert org-dp-create template.
| 
| [back]
`----

Example: you will be prompted for the org element (in this case I
decided to use "example-block"), you don't have to type anything, just
do this single selection when prompted.

,----
| ;; Affiliated keywords: '(:kw1 val1 :kw2 val2 ...)
| ;; :name "val"
| ;; :plot "val"
| ;; :results ("val") or ("val" "key")
| ;; :header ("val") or ("val1" "val2")
| ;; :caption (("val")) or (("val" "key")) or
| ;;          (("val2" "key2") ("val2" "key2"))
| ;; :attr_xyz ("val") or ("val1" "val2")
| (org-dp-create 'example-block nil t ;cont ins
| nil ;aff 
| :switches ""
| :preserve-indent ""
| :value ""
| )
`----

What you see: you can just write "Hello World" in the 

,----
| :value ""
`----

parameter and will get an example block with "Hello World" as content
(funny enough, some org elements have 'value' as their content, others
'content').  Check the docstring of org-dp-create for more info on its
args.  You can add affiliated keywords too, as demonstrated (as one
function arg).


REWIRE:

,----[ C-h f tempo-template-org-dp-rewire RET ]
| tempo-template-org-dp-rewire is an interactive Lisp function.
| 
| (tempo-template-org-dp-rewire &optional ARG)
| 
| Insert org-dp-rewire template.
| 
| [back]
`----

or this one, if you don't just want to set properties, but need a lambda
function to do work in calculation the new property (with access to the
old property and the whole plist of the old element you want to
rewire/modify):

,----[ C-h f tempo-template-org-dp-rewire-lambda RET ]
| tempo-template-org-dp-rewire-lambda is an interactive Lisp function.
| 
| (tempo-template-org-dp-rewire-lambda &optional ARG)
| 
| Insert org-dp-rewire template with lambdas.
| 
| [back]
`----

Examples:

,----
| (org-dp-rewire 'example-block nil t ;cont ins
| nil ;aff 
| nil ;elem 
| :switches ""
| :preserve-indent ""
| :value ""
| )
`----

,----
| (org-dp-rewire 'example-block nil t ;cont ins
| nil ;aff 
| nil ;elem 
| :switches '(lambda (old elem)  )
| :preserve-indent '(lambda (old elem)  )
| :value '(lambda (old elem)  )
`----

Again, check the docstring of org-dp-rewire for more info on its args.

I hope this helps already, feel free to ask any further questions (maybe
put me in Cc since I'm not a very frecuent visitor of the list).

If I find time and motivation I might look at your specific example/use
case, but I cannot promise that.

-- 
cheers,
Thorsten