Re: [O] table.el complex tables and orgtbl-to-latex

[Nicolas Goaziou]

Hello,

Carsten Dominik <address@hidden> writes:

> Hi Uwe,
>
> this sounds interesting - would you be interested to provide a patch
> to this effect?

Somewhere in one of my local branches, I have started replacing current
table export functions with export framework.

Basically, the idea is to generate a temporary export back-end on the
fly according to parameters provided in the header.
`my-orgtbl-to-generic'[fn:1] should work (although I didn't implement
tests yet) but there's one missing part. Indeed, it still relies on
`orgtbl-to-orgtbl' to convert table from lisp (as returned by
`org-table-to-lisp') to regular Org syntax.

So, it only needs a function doing the conversion that would not call
`orgtbl-to-generic', which should be pretty straightforward.

Once this is done, export back-ends still need some slight changes: for
example `latex' needs a way to get passed a value for
`org-latex-table-scientific-notation' through an external plist, which
boils down to adding an entry for this variable in back-end
definition[fn:2].

[fn:1] (defun my-orgtbl-to-generic (table params)
    "Convert the orgtbl-mode TABLE to some other format.

  This generic routine can be used for many standard cases.

  TABLE is a list, each entry either the symbol `hline' for
  a horizontal separator line, or a list of fields for that
  line.  PARAMS is a property list of parameters that can
  influence the conversion.

  Valid parameters are:

  :backend

    Export back-end used to transcode cells.  Default is `org'.
    The value has to refer to an already registered back-end.

  :splice

    When set to t, return only table body lines, don't wrap them
    into :tstart and :tend.  Default is nil.  When :splice is
    non-nil, this also means that the exporter should not
    interpret header and footer sections.

  :hline

    String to be inserted on horizontal separation lines. May
    be nil to ignore hlines.

  :sep

    Separator between two fields, as a string.

  Each in the following group may be either a string or a function
  of no arguments returning a string:

  :tstart

    String to start the table.  Ignored when :splice is t.

  :tend

    String to end the table.  Ignored when :splice is t.

  :lstart

    String to start a new table line.

  :llstart

    String to start the last table line, defaults to :lstart.

  :lend

    String to end a table line

  :llend

    String to end the last table line, defaults to :lend.

  Each in the following group may be a string or a function of one
  argument (the field or line) returning a string:

  :lfmt

    Format for entire line, with enough %s to capture all fields.
    If this is present, :lstart, :lend, and :sep are ignored.

  :llfmt

    Format for the entire last line, defaults to :lfmt.

  :fmt

    A format to be used to wrap the field, should contain %s
    for the original field value.  For example, to wrap
    everything in dollars, you could use :fmt \"$%s$\".  This
    may also be a property list with column numbers and
    formats.  For example :fmt (2 \"$%s$\" 4 \"%s%%\")

  :hlstart :hllstart :hlend :hllend :hlsep :hlfmt :hllfmt :hfmt

   Same as above, specific for the header lines in the table.
   All lines before the first hline are treated as header.  If
   any of these is not present, the data line value is used.

  This may be either a string or a function of two arguments:

  :efmt

    Use this format to print numbers with exponential.  The
    format should have %s twice for inserting mantissa and
    exponent, for example \"%s\\\\times10^{%s}\".  This may
    also be a property list with column numbers and formats.
    :fmt will still be applied after :efmt.

  In addition to this, the parameters :skip and :skipcols are always handled
  directly by `orgtbl-send-table'.  See manual."
    (require 'ox-org)
    (let* ((backend (plist-get params :backend))
           (splice (plist-get params :splice))
           (tstart (plist-get params :tstart))
           (tend (plist-get params :tend))
           (efmt (plist-get params :efmt))
           (fmt (plist-get params :fmt))
           (lfmt (plist-get params :lfmt))
           (llfmt (or (plist-get params :llfmt) lfmt))
           (hfmt (or (plist-get params :hfmt) fmt))
           (hlfmt (or (plist-get params :hlfmt) lfmt))
           (hllfmt (or (plist-get params :hlfmt) llfmt))
           (lstart (plist-get params :lstart))
           (llstart (or (plist-get params :llstart) lstart))
           (hlstart (or (plist-get params :hlstart) lstart))
           (hllstart (or (plist-get params :hllstart) lstart))
           (lend (plist-get params :lend))
           (llend (or (plist-get params :llend) lend))
           (hlend (or (plist-get params :hlend) lend))
           (hllend (or (plist-get params :hllend) lend))
           (sep (plist-get params :sep))
           (hlsep (or (plist-get params :hlsep) sep))
           (hline (plist-get params :hline)))
      (org-export-string-as
       (orgtbl-to-orgtbl table nil)
       ;; Build a temporary back-end from PARAMS.
       (org-export-create-backend
        :parent (cond ((not backend) 'org)
                      ((not (org-export-get-backend backend))
                       (user-error "Unknown :backend value"))
                      (t backend))
        :transcoders
        (list
         (when (or splice tstart tend)
           (cons 'table
                 `(lambda (table contents info)
                    ,(if splice 'contents
                       `(concat ,(cond ((null tstart) nil)
                                       ((functionp tstart)
                                        (concat (funcall tstart) "\n"))
                                       ((stringp tstart) (concat tstart "\n"))
                                       (t (user-error "Wrong :tstart value")))
                                contents
                                ,(cond ((null tend) nil)
                                       ((functionp tend) (funcall tend))
                                       ((stringp tend) tend)
                                       (t (user-error "Wrong :tend 
value"))))))))
         (when (or splice hllfmt hlfmt llfmt lfmt lstart llstart hllstart lend 
llend hllend)
           (cons
            'table-row
            `(lambda (row contents info)
               (if (eq (org-element-property :type row) 'rule) ,hline
                 (let* ((headerp (= (org-export-table-row-group row info) 1))
                        (lastp (not (org-export-get-next-element row info)))
                        (last-in-header-p
                         (and headerp
                              (org-export-table-row-ends-rowgroup-p row info))))
                   (cond
                    ((or (not contents) ,(and splice 'headerp)) nil)
                    ;; Check if we can apply `:lfmt', `:llfmt', `:hlfmt',
                    ;; or `:hllfmt' to CONTENTS.  Otherwise, fall-back on
                    ;; `:lstart', `:lend' and their relatives.
                    ,(when hllfmt
                       `(last-in-header-p
                         ,(cond ((functionp hllfmt) `(funcall ,hllfmt contents))
                                ((stringp hllfmt) `(format ,hllfmt contents))
                                (t (user-error "Wrong :hllfmt value")))))
                    ,(when hlfmt
                       `(headerp
                         ,(cond ((functionp hlfmt) `(funcall ,hlfmt contents))
                                ((stringp hlfmt) `(format ,hlfmt contents))
                                (t (user-error "Wrong :hlfmt value")))))
                    ,(when llfmt
                       `(lastp
                         ,(cond ((functionp llfmt) `(funcall ,llfmt contents))
                                ((stringp llfmt) `(format ,llfmt contents))
                                (t (user-error "Wrong :llfmt value")))))
                    (t ,(cond ((not lfmt)
                               `(concat (or (and last-in-header-p ,hllstart)
                                            (and lastp ,llstart)
                                            ,lstart)
                                        contents
                                        (or (and last-in-header-p ,hllend)
                                            (and lastp ,llend)
                                            ,lend)))
                              ((functionp lfmt) `(funcall ,lfmt contents))
                              ((stringp lfmt) `(format ,lfmt contents))
                              (t (user-error "Wrong :lfmt value"))))))))))
         (when (or efmt fmt sep hlsep)
           (cons
            'table-cell
            `(lambda (cell contents info)
               (let* ((column
                       (1+ (cdr (org-export-table-cell-address cell info))))
                      (row (org-export-get-parent-element cell))
                      (headerp (= (org-export-table-row-group row info) 1))
                      (lastp (not (org-export-get-next-element row info))))
                 (when contents
                   ;; Check if we can apply `:efmt' on CONTENTS.  If
                   ;; `:efmt' binds columns to format strings or
                   ;; functions, first get the right one.
                   ,(when efmt
                      `(when (string-match orgtbl-exp-regexp contents)
                         (let ((mantissa (match-string 1 contents))
                               (exponent (match-string 2 contents)))
                           ,(cond
                             ((stringp efmt)
                              `(setq contents (format ,efmt mantissa exponent)))
                             ((functionp efmt)
                              `(setq contents (funcall ,efmt mantissa 
exponent)))
                             ((consp efmt)
                              `(let ((efmt (cadr (memq column ',efmt))))
                                 (cond
                                  ((null efmt) nil)
                                  ((stringp efmt)
                                   (setq contents (format ,efmt mantissa 
exponent)))
                                  ((functionp efmt)
                                   (setq contents (funcall ,efmt mantissa 
exponent)))
                                  (t (user-error "Wrong :efmt value")))))
                             (t (user-error "Wrong :efmt value"))))))
                   ;; Check if we can apply :fmt on CONTENTS.  If `:fmt'
                   ;; binds columns to format strings or functions, first
                   ;; get the right one.
                   ,(cond
                     ((null fmt) nil)
                     ((stringp fmt) `(setq contents (format ,fmt contents)))
                     ((functionp fmt) `(setq contents (funcall ,fmt contents)))
                     ((consp fmt)
                      `(let ((fmt (cadr (memq column ',fmt))))
                         (cond
                          ((null fmt) nil)
                          ((stringp fmt) (setq contents (format fmt contents)))
                          ((functionp fmt) (setq contents (funcall fmt 
contents)))
                          (t (user-error "Wrong :fmt value")))))
                     (t (user-error "Wrong :fmt value"))))
                 ;; Return transcoded cell, maybe with a separator.
                 (concat contents
                         (and (not (or ,lfmt
                                       (and headerp ,hlfmt)
                                       (and lastp ,hllfmt)))
                              (org-export-get-next-element cell info)
                              (or (and headerp ,hlsep) ,sep)))))))))
       'body-only '(:with-tables t))))

[fn:2] 

  (defun my-orgtbl-to-latex2 (table params)
    "Convert the orgtbl-mode TABLE to LaTeX.

  TABLE is a list, each entry either the symbol `hline' for
  a horizontal separator line, or a list of fields for that
  line.  PARAMS is a property list of parameters that can
  influence the conversion.  Supported parameters are:

  :splice

    When set to t, return only table body lines, don't wrap
    them into a tabular environment.  Default is nil.

  :fmt

    A format to be used to wrap the field, should contain %s
    for the original field value.  For example, to wrap
    everything in dollars, use :fmt \"$%s$\".  This may also
    be a property list with column numbers and formats.  For
    example

      :fmt (2 \"$%s$\" 4 \"%s%%\")

    The format may also be a function that formats its one
    argument.

  :efmt

    Format for transforming numbers with exponentials.  The
    format should have %s twice for inserting mantissa and
    exponent, for example \"%s\\\\times10^{%s}\".  Default
    value is defined in `org-latex-table-scientific-notation'.

  The general parameters :skip and :skipcols have already been
  applied when this function is called."
    (require 'ox-latex)
    (let ((splice (plist-get params :splice))
          (fmt (plist-get params :fmt))
          (efmt (plist-get params :efmt)))
      (org-export-string-as
       (orgtbl-to-orgtbl table nil)
       (org-export-create-backend
        :parent 'latex
        :transcoders
        (list
         (and splice '(table . (lambda (table contents info) contents)))
         (and fmt
              (cons
               'table-cell
               `(lambda (cell contents info)
                  (let* ((contents
                          (when contents
                            ,(cond
                              ((stringp fmt) `(format ,fmt contents))
                              ((functionp fmt) `(funcall ,fmt contents))
                              ((consp fmt)
                               `(let* ((column
                                        (1+ (cdr (org-export-table-cell-address
                                                  cell info))))
                                       (fmt (cadr (memq column ',fmt))))
                                  (cond
                                   ((null fmt) contents)
                                   ((stringp fmt) (format fmt contents))
                                   ((functionp fmt) (funcall fmt contents))
                                   (t (user-error "Wrong :fmt parameter")))))
                              (t (user-error "Wrong :fmt parameter"))))))
                    (org-export-with-backend 'latex cell contents info)))))))
       'body-only
       (nconc (list :with-tables t :latex-table-centered nil)
              (and efmt (list :latex-table-exponentials efmt))))))


Regards,

-- 
Nicolas Goaziou