Re: python-mode: make sure output is not eaten

[Stefan Monnier]

> #> > Have you installed the complete patch? Because current version of the
> #> > CVS code doesn't quite work for me... it hangs emacs whenever I try
> #> > python-send-buffer (I need C-g to get it make it responsive again),
> #> > which seems to be related to the "while" loop in python-send-receive
> #> > never finishing due to the (local-variable-p 'python-preoutput-result)
> #> > in python-preoutput-filter never being true...
> #> 
> #> Hmm... looks like the two pieces of code don't run in the same buffer
> #> maybe?  I've just installed a patch which may fix it.

> I works better -- no longer hangs my Emacs, but after python-send-buffer
> the final prompt still doesn't appear.

I believe my last patch fixes this as well.

> Agreed. But why is removing "_emacs_out ()" any more selective than
> removing ">>> _emacs_out ()"?

We're talking about "^_emacs_out " vs "^\\(>>> \\)?_emacs_out": clearly one
of the two regexps describes a superset of the other.  If you can guarantee
that it'll always match "^>>> _emacs_out ", then it's a different question.

> I think I understand your reasoning, but I do not think I agree with it.
> If we *want* to wait for prompt after both "import ..." and "print ...",
> we can always use python-send-receive twice, instead of
> python-send-string followed by python-send-receive.  But I still do not
> see why one is supposed to be more robust than the other.

python-send-receive doesn't wait for a prompt but for the _emacs_out cookie,
which the `import' does not output.

> Also, I have finally found a case in which adding "; print ..." breaks
> things: assume the string is "if False: print 1". Observe that "if
> False: print 1; print '_emacs_out ()'" does *not* print anything.

> It may not be a *real* problem, but it does -- IMHO -- show that sending
> two separate commands *is* more robust.

It's not the only case, and I agree it's not a purely academic question.
But: such cases does not occur in python.el (where we only add the
"; print ..." at a few special spots where we have control over the text
that precedes it).

> #> > I do not understand why "command" python-send-command shouldn't
> #> > contain newline...
> #> 
> #> It's not a command, so it can only called from elisp code. Check the
> #> callers and you'll see it is never called with a newline. So it's OK
> #> to make such an assumption.

> Well, maybe. But still, either python-send-string works for multiline
> strings (in which case python-send-command should work as well)

I see no reason why the two should be so linked.

> or python-send-string doesn't work for multiline strings, in which case
> the assert in python-send-command doesn't help at all (as you wrote,
> nothing can call it with multiline command anyway).

In python-send-string, multiline strings work.  In python-send-command
multiline strings were very likely to fail because we added "; print ...".
Hence the presence of an `assert' (assertions are of no use to the user,
their only meaning is for the programmer as a form of documentation that's
slightly better sync'd with the code than mere comments).

> #> > It should work just fine with multiple lines, the only thing which
> #> > might cause problems is indentation, and even that only if there is
> #> > an indented block at the very end.
> #> 
> #> Sending a multiline thingy via python-send-string will look like
> #> absolute crap in the resulting output,

> I disagree...

Actually, you seem to be right: the prompt-elimination heuristic seems to
work well for this in practice.

Although it points top another problem: the result of the evaluation is
printed on the line of the prompt.  I.e. if I start with

   >>> 

after python-send-string "print 1" I get

   >>> 1
   >>> 

rather than

   >>> 
   1
   >>> 

or

   >>> print 1
   1
   >>> 

> #> because all the intermediate prompts will get concatenated rather
> #> than interleaved with the input.

> Right, but what's wrong with that? If I send a string like "a = 1\n
> print 'Test'\n b = 2", what's wrong with getting output like
> ">>> Test\n>>> \n >>>"?

In a session log, the prompt is expected to be followed by user input, not
program output.

> #> So I see no point in trying to support this in a case such as
> #> `python-send-command' where we don't use this flexibility anyway (and
> #> where we need to analyze the output, so the more control over it the
> #> better).

> You may be right... but if you think it makes sense, the assert should
> be in python-send-string, not in python-send-command.

I see no point in restricting the use of python-send-string where the only
potential problem is ugly output, but not incorrect execution.  Whereas in
python-send-command, the result could have been incorrect execution.
Also adding the assert to python-send-command did not restrict the user in
any way, only the python.el hacker.

> #> > I do not think this is the right way to solve the problem. IMHO, the
> #> > variable should be called python-inhibit-output and should prevent
> #> > *anything* from being printed in Inferior Python buffer -- thus,
> #> > functions like eldoc could bind it to 't and we would not need to worry
> #> > about them at all.
> #> 
> #> > After all, it doesn't matter how the output of python-eldoc-function
> #> > looks like, it should never appear in the comint buffer. And it would
> #> > let us simplify the common case of *user* wanting to run Python code.
> #> 
> #> Sounds good. As long as you can be sure that there can be no other
> #> output at the same time (i.e. the python process is idle, waiting at
> #> the prompt when you send the eargs command). Currently, we do not
> #> check that python is idle, so we have no guarantee that we can just
> #> discard all the output between after we send the `eargs' (ou modpath,
> #> or complete, ...) until after we see the _emacs_out.

> I have hard time imagining how the current code could work at all if it
> send anything to Python interpreter while it was busy, but I will take a
> closer look at this.

Indeed, if the python process is busy when we send the command, the result
may be wrong.  But I expect that in most cases it'll be right.  Still,
I think checking busy/idle-ness is a good idea in itself and your above
suggestion as well.

Note that checking idle/busy-ness can be tricky.  Even assuming the
program's output does not look like a prompt, you may have to count the
number of newlines sent to the python's process to know how many prompts to
expect in return.  Or alternatively, you'd have to send each input line one
at a time, waiting for the prompt before sending the next one.


        Stefan