Re: [O] Proposal and RFC for improving ob-python

[Kyle Meyer]

Ondřej Grover <address@hidden> writes:

[...]

>> But what about when :results != value?  Doesn't your proposal only
>> handle returning the last value?
>>
> You mean :results output ? In that case it could just omit the
> "open(...).write(...) " part and capture anything the console prints before
> the primary prompt appears again.
> Or it could capture stdout into a file, but I think that's needlessly
> complicated:
>
> python -i << HEREDOC_END
> import sys
> sys.stdout = open(<TMP FILE or PIPE>)   # replace stdout with some file
> _ = block_eval("""
> <CODE BLOCK BODY>
> """)
> sys.stdout.close()
> sys.stdout = sys.__stdout__  # restore stdout
> HEREDOC_END

(I think you should rely on Org babel's shared utilities,
e.g. org-babel-eval and org-babel-eval-read-file, unless there is a
specific reason they won't do.  At any rate, right now I'm just trying
to get a picture of how block_eval would fit in.)

So these are the main options to consider for Python:

    |             | output | value |
    |-------------+--------+-------|
    | non-session | a      | b     |
    | session     | c      | d     |

a) Using block_eval here seems unhelpful here because the method in
   place is already simple:

   (org-babel-eval org-babel-python-command body)

b) block_eval replaces the need for dumping the code into a function.
   Using block_eval in org-babel-python-wrapper-method would allow you
   to get rid of the odd "return" use.

c) Currently, this is handled by inserting the body lines one at a time
   and then getting the output by splitting on the prompt.  Aside from
   race condition issues, this has limitations with handling blank lines
   in the body.

   block_eval could help here because the whole body could be sent
   through as one line of input to the prompt.  This should resolve most
   issues with prompt splitting, as well as lift the body formatting
   restrictions for sessions.

   However, the main issue I see with this is that I think it would lose
   the main distinction between session and non-session for ":results
   output" that is discussed in (info "(org) Results of evaluation"):

       #+BEGIN_SRC python :results output
        print "hello"
        2
        print "bye"
       #+END_SRC

       #+RESULTS:
       : hello
       : bye

   versus this

       #+BEGIN_SRC python :results output :session
        print "hello"
        2
        print "bye"
       #+END_SRC

       #+RESULTS:
       : hello
       : 2
       : bye

d) The variable _ is assigned the last value by the shell, so the
   current mechanism should work even if block_eval is used as described
   in c.

I've attached a quick-and-dirty patch, just for discussion.  At least
under light testing, it resolves the issues I listed in my previous
response [1].  It does have the change in behavior discussed in c, which
I see as problematic.

It also raises the issue of how to incorporate the block_eval code.  I
think there are two main options.

* Require the user have the module installed and import it.

* Make the module into a snippet to go into the ob-python.el.

The first option is what the current patch does.  The problem is that it
requires users to install an external module.  The second option avoids
that but is uglier and harder to maintain.


[1] There's a remaining issue, which I believe is specific to newer
    versions of python.el, where the first time a session block is
    executed, it doesn't wait for the shell to load up before sending
    input.

0001-WIP-lisp-ob-python.el-Use-block_eval.patch
Description: Text Data

--
Kyle