bug#23906: 25.0.95; Undo boundary after process output is not consistent

[Markus Triska]

Hi Phillip, Stefan,

I post a bit of information about how ediprolog.el currently works so
that the use case is clear. I am perfectly willing to make any
adjustments to ediprolog so that undo is treated more consistently. At
the same time, I am also interested in improving Emacs generally, so
please read this as clarification of, not excuse for, the current state.

phillip.lord@russet.org.uk (Phillip Lord) writes:

>> The precise sequence is (see `ediprolog-query' and `ediprolog-interact'):
>>
>> 1) "\n%@ " is inserted into the buffer
>> 2) the Prolog process is started (if necessary)
>> 3) the query is sent to the process
>> 4) when process output arrives, it is appended to the buffer.
>>
>> The buffer is modified only by (1) and (4). Somtimes there is an undo
>> boundary between them, and most often there is none (which I prefer).
>
> Yes. So the issue here is that between 1) and 4) there is a
> indeterminate length of time.

Yes, this is the core issue. In simulans2.sh which I posted, you can try
different time spans between 1) and 4) easily by modifying line 15.  For
example, if I change 1.5 to 10, I always get an undo boundary after the
initial %@. If I change it to 4 or 5, I still only sometimes get one. My
estimate is thus that the timer runs about once every 10 seconds ;-)


>> That's exactly what I expect to happen. But in some cases, there
>> unexpectedly *is* an undo boundary between two subsequent insertions of
>> text into the buffer, namely between (1) and (4) above.
>
> The problem here also is what happens if the process output is long.
> Emacs will now add undo-boundaries periodically. Before, I think, the
> undo list would have got longer and longer, until emacs errored.

To clearify, the use case of ediprolog indeed comprises a single flow of
insertions, either stemming from process output or user input, while the
Prolog query is in process: Everything the Prolog process outputs is
inserted. Everything the user inputs is also inserted into the buffer
(and sent to the process). The result is a complete transcript of the
Prolog interaction just as it would normally happen on the terminal.

For this particular case, I would be perfectly OK if undo would simply
undo the whole flow of insertions up to the point where the query was
started. I do not know much about the undo representation or your plans
about it, but what I think would work in this particular case is to
record it as "Emacs, to undo this, you need to remove all text between X
and Y", and Y simply increases as more text is inserted into the buffer.
This would keep the length of the undo list constant throughout the
whole interaction with the subprocess, at least in this concrete
case. Can I do this in ediprolog myself? If so, then I will do it.

> Yes. Or rather you could insert %@ immediately before getting data back
> from prolog, perhaps with a process filter. Then both of your insertions
> would be happening together.

I am already using a process filter: If the process outputs \n, then I
insert \n%@ into the buffer. Only for the very first %@ that I insert
directly after the query is started, I am currently doing it outside the
process filter. It is important to insert %@ immediately to give an
indication that the query has been sent to the process (which may take a
long time to produce output). Note that % starts a Prolog line comment,
so the process output that gets embedded in buffers does not interfere
with the Prolog program itself. For this reason, text properties would
not be a good fit for this use case: Users often want to store the query
results together with the source code, and using %@ allows that.

> Also, what happens if point moves between 1) and 4)?

In the typical interaction mode, point cannot move between 1) and 4):
Each key press is sent to the Prolog process, so that you can interact
with Prolog like in a terminal. However, it is indeed possible to escape
this loop with C-g and indeed move point and even insert text into the
buffer while the process still runs and may produce output that is also
inserted into the buffer by the process filter.

> Stefan, any thoughts?

As to the options outlined by Stefan:

   1- just let the user hit undo an extra time when that happens.

That is the current state that I consider a bit unsatisfying, so I would
like to choose between 2, 3 and 4:

   2- Refine the timer-based undo-boundaries along the lines of what you
      had earlier: only add a timer-based boundary if the last undo chunk
      is too long (same for undo-boundaries added in non-current buffers).
      It would still occasionally add "unwanted" boundaries, but only when
      not doing so would result in too large an undo list.

I think we can rule this out as a satisfying solution.

   3- Have Ediprolog use the same trick used in Viper where we wipe out
      intermediate boundaries after the fact.

This sounds like a great option. Could you please tell me a bit more?
The specific use case for me is: I have a process filter that
continuously (upon process output or user input) inserts text into the
buffer. I just want *all* this text gone when pressing C-/. If possible,
I hope that this can be done without the undo list growing too long.

   4- Add the kind of "do it manually" option you had added earlier, such
      that Ediprolog could request that Emacs refrain from auto-adding any
      undo-boundaries in its buffer.

To me, this sounds at least as great as (3) and maybe could be used by
both Viper and ediprolog if it were available?

Please note in this case that, as explained above, the output normally
is inserted continuously in a loop, but it is possible to break out of
the loop with C-g and edit text elsewhere in the buffer, and in that
case I still would like the normal undo behaviour for user
input. Ideally, only the output that stems from the interaction with the
Prolog process and user input during the interaction should be treated
as a single unit. This separation (i.e., treat insertions as a unit in
some parts of a buffer while applying the normal undo behaviour in other
parts) may of course be very hard to achieve, and I would already be
very satisfied if there is a way to make it work in the regular case. I
only mention for completeness that conceptually, I regard the text that
stems from the Prolog interaction different from editing operations.

Please try ediprolog if possible (any SWI version will do) and try for
example the query (M-x ediprolog-dwim RET with point on the line):

?- length(Ls, L).

You then interact with SWI-Prolog just as on a terminal. C-g will put
you out of the loop and let you edit other text in the buffer.
M-x ediprolog-toplevel RET puts you back to the toplevel.

You can stop the query at any time by pressing "." or "a". Ideally,
pressing C-/ after stopping the query undoes the whole interaction.

I hope that this clearifies the use case a bit.

Thank you for looking into this!

All the best,
Markus