Re: [O] Export arrays for 'sh' code blocks when using bash

[Pascal Fleury]

Hi Eric, see comments inline.

On Thu, Mar 27, 2014 at 6:43 PM, Eric Schulte <address@hidden> wrote:

Hi Pascal,

This looks like a good patch.  If I could make three changes/requests.

 

Thanks, I am happy to see it makes sense.

 

1. This patch should also work for "begin_src bash" code blocks.  After
   a very quick glance it doesn't appear to.

I have actually tried this with the latest org-mode release (8.2.5h) and it did plainly not accept 'begin_src bash'. Maybe that's new with the ob-shell.el (I noticed it has been renamed :-)

 

2. Could you include one or two tests ensuring that this patch does what
   it intends with bash code blocks, and does not affect code blocks
   executed with bin/sh?

Will do, seeing it has some traction.

 

3. Large contributions like this require some FSF paperwork.  Please see
   the following page for information on requirements to contribute.

I checked internally, and apparently, Google has already such a signed document. It would run on the same document, as I am a google employee.

 

   http://orgmode.org/worg/org-contribute.html

Thanks, and I look forward to seeing this merged into Org-mode!
Eric

Pascal Fleury <address@hidden> writes:

> Hello,
>
> I'dl like to propose a patch for inclusion into org-mode (ob-shell.el in
> particular).
>

> *TL;DR:* use arrays and associative arrays when exporting variables to bash
> in 'sh' code blocks.
>
> *Details:*

> When variables are defined in a 'sh' code block, they are exported as
> strings. when the variable itself is an array or a table, then we simply
> get a shell variable that contains the list of all values in a
> non-structured form. E.g.
>
> #+NAME: my_list
> | one   |
> | two   |
> | three |
>
> #+NAME: experiment
> | name | first_attempt    |
> | date | [2014-03-27 Thu] |
> | user | fleury           |
>
> #+BEGIN_SRC sh :var scalar="some value" :var array=my_list :var table=config
> echo ${scalar}  # -> prints 'some value'
> echo ${array}   # -> prints 'one two three'
> echo ${table}   # -> prints 'first attempt [2014-03-27 Thu] fleury'
> #+END_SRC
>
> This will print simple strings. Also, there is no easy way to access the
> date of the experiment, for example. Now bash has things like arrays and
> associative arrays, but the current ob-shell.el does not use these.
> Probably because their syntax is bash-specific, and ob-shell.el is
> shell-agnostic.
>
> My patch (attached) changes this in the case you have
> (setq org-babel-sh-command "bash")
> in your emacs config somewhere. If any other value is used, it continues to
> export them as we do today (I don't know enough about other shells).
>
> In that case, it will export the list as an array, the table as an
> associative array, and the scalar as it does already. So the 'sh' code
> block could then use
>
> #+BEGIN_SRC sh :var scalar="some value" :var array=my_list :var table=config
> echo ${scalar}
> echo ${array[1]} # -> prints "two"
> echo ${table[user]} # -> prints "fleury"
> #+END_SRC
>
> In the case we have a bigger table, then the first row is used as key, the
> others are represented as a string with all the values (as it is for array
> currently). bash does not have multi-dimensional arrays, so it needs to be
> a string.
>
> This makes writing shell snippets much easier in my experience, and there
> I'd like to propose this fix for the org-mode community at large.
>
> --paf
>

> diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el
> index 3ede701..0a691e2 100644
> --- a/lisp/ob-shell.el
> +++ b/lisp/ob-shell.el
> @@ -106,6 +106,30 @@ This function is called by `org-babel-execute-src-block'."
>
>  ;; helper functions
>
> +(defun org-babel-variable-assignments:bash_array (varname values &optional sep hline)
> +  "Returns a list of statements declaring the values as a bash array."
> +  (format "declare -a %s=( \"%s\" )"
> +     varname
> +     (mapconcat 'identity
> +       (mapcar
> +         (lambda (value) (org-babel-sh-var-to-sh value sep hline))
> +         values)
> +       "\" \"")))
> +
> +(defun org-babel-variable-assignments:bash_associative (varname values &optional sep hline)
> +  "Returns a list of statements declaring the values as bash associative array."
> +  (format "declare -A %s\n%s"
> +    varname
> +    (mapconcat 'identity
> +      (mapcar
> +        (lambda (items)
> +          (format "%s[\"%s\"]=%s"
> +            varname
> +            (org-babel-sh-var-to-sh (car items) sep hline)
> +            (org-babel-sh-var-to-sh (cdr items) sep hline)))
> +        values)
> +      "\n")))
> +
>  (defun org-babel-variable-assignments:sh (params)
>    "Return list of shell statements assigning the block's variables."
>    (let ((sep (cdr (assoc :separator params)))
> @@ -114,9 +138,17 @@ This function is called by `org-babel-execute-src-block'."
>                    "hline"))))
>      (mapcar
>       (lambda (pair)
> -       (format "%s=%s"
> -            (car pair)
> -            (org-babel-sh-var-to-sh (cdr pair) sep hline)))
> +       (if (and (string= org-babel-sh-command "bash") (listp (cdr pair)))
> +         (if (listp (car (cdr pair)))
> +           (org-babel-variable-assignments:bash_associative
> +           (car pair) (cdr pair) sep hline)
> +           (org-babel-variable-assignments:bash_array
> +           (car pair) (cdr pair) sep) hline)
> +         (format "%s=%s"
> +         (car pair)
> +         (org-babel-sh-var-to-sh (cdr pair) sep hline))
> +       )
> +     )
>       (mapcar #'cdr (org-babel-get-header params :var)))))
>
>  (defun org-babel-sh-var-to-sh (var &optional sep hline)

--
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

--
--paf