Re: Aborting display. Is this possible?

[Alan Mackenzie]

Hello, Eli.

On Mon, Oct 20, 2014 at 06:12:05PM +0300, Eli Zaretskii wrote:
> > Date: Mon, 20 Oct 2014 11:09:49 +0000
> > Cc: address@hidden
> > From: Alan Mackenzie <address@hidden

> > On a large C Mode buffer (such as xdisp.c), hold down the PageDown key
> > for, say, 10 seconds.

> > What is seen: The screen freezes.  After releasing the key, it takes many
> > seconds (~10) before the current buffer position is displayed on the
> > screen.

Incidentally, I saw this happening today in emacs-lisp-mode, on my
terminal with a 3 column follow-mode.

> > What I'd like to see: The screen continuously updating with (not
> > necessarily contiguous) parts of the buffer, the updating being frequent
> > enough to give the illusion of continuous scrolling.  On releasing the
> > key, the screen "instantly" displaying the new buffer position.

> I understand the desire, and I'm not saying it isn't reasonable.  So
> please don't take what's below as a rebuttal; I'm just trying to
> explain how the current Emacs design works against your wishes, and
> what could/should be changed for your wishes to maybe become reality.

> I will probably say some (or many) things that you already know, so
> please bear with me.

Thanks for such a clear exposition.

> First, Emacs's basic design is that it processes all the available
> input before it enters redisplay.  As long as there's input in the
> input queue, Emacs will not call redisplay.  (There are 2 exceptions
> to this rule: processing input could indirectly call the 'redisplay'
> function; and messages in the echo area produced while processing
> input require a kind of "partial" redisplay to show the echo area.)
> Why? because processing input allows to avoid showing the user a
> screen that is outdated from the user's POV, given that input more
> often than not changes what should be displayed.

Up to now, I think I've been confusing display with font-locking.
Obviously f-l has to happen when d is being done, but not all
font-locking is caused by display.

> Another part of this puzzle is that redisplay includes fontification
> of the visible portion of the buffer.  Emacs applies the font-lock
> faces (and any other faces) during the 1st stage of its redisplay
> cycle, when it computes the so-called "glyph matrices", which are data
> structures maintained for each window that describe what the updated
> screen should look like.  As David points out, it is impossible to
> compute the glyph matrices without applying the faces, because faces
> change the metrics of each character, and thus affect the layout,
> which is what glyph matrices are all about.

> The final part of the puzzle is what the scroll-up-command does when
> Emacs processes a single PageDown keypress.  It computes the new
> starting point for the window, which (if we neglect complications) is
> simply the beginning of line N screen lines below the original window
> start, where N is almost the height of the window.  (Note that I wrote
> "screen lines", not "physical lines", so this is not entirely trivial
> when there are continuation lines and variable-size fonts.)  Then it
> determines whether this change of the window-start requires to move
> point to bring it into the view, and if so, moves point as well.

On a PageDown, Emacs needs to do font-locking just to work out how far
awy the next page is, like David said.  There is a comment in
window_scroll_line_based: "Fvertical_motion enters redisplay, which can
trigger fontification...".  This is the killer: scrolling a screen of
xdisp.c is taking, on average, ~0.09s, the bulk of this time being
fontification, whereas the key events are coming in every ~0.03s.
Some way has to be found to scroll with only every third screen (at most)
getting font-locked.

If every glyph were assumed to be the same size, the calculation of the
window-start position for the current scroll operation would be _much_
faster: display would still have to account for text lines overflowing
screen lines, and invisible text, and so on, 

So, how about the following strategy: when the (new) variable
`face-instead-of-fontifying' is bound to a face, AND the input queue is
non-empty (or, perhaps, very non-empty), the appropriate subroutine of
window-scroll should deem characters without a `face' (or `fontified' ?)
property to have face `face-instead-of-fontifying'.

This should empty the input queue pretty quickly, enabling a current
buffer portion to get displayed frequently enough.

> So, after processing a single PageDown key, the window has a new
> starting point, and point in that window's buffer has been moved.

> If, after processing this single PageDown key, the input queue is
> empty (as it is when you lean on the key, because Emacs finishes the
> above processing in less time than the auto-repeat interval), Emacs
> enters redisplay.  Redisplay computes the glyph matrix of the window
> given its start point as set by scroll-up-command; as part of this
> matrix computation, the display engine applies all the font-lock
> properties to the text, and sets the 'fontified' property on that
> text.  Then, just before it is about to redraw the window given the
> new glyph matrix, it checks for input.  If there's input available,
> and redisplay-dont-pause is nil, the window will not be redrawn,
> i.e. this redisplay cycle will be aborted.

But in the default situation (redisplay-dont-pause is non-nil), once the
calculation of the glyph matrix has begun, the redrawing necessarily
takes place.  This is good.  :-)

> Then the same will happen again for the next PageDown key.  The result
> is a frozen window display, because all the attempts to redisplay are
> aborted due to input rate that is high enough to preempt redisplay,
> but not high enough to prevent Emacs from entering redisplay after
> every (or almost every) PageDown key.  To see more frequent updates
> that give an illusion of scrolling, we would need a feature that
> ignores the input availability, or maybe dynamically changes the
> number of input events that could be ignored when redisplay falls
> behind.  We don't have such a feature.

Is this really the case?  Is it not that the font-locking caused by
scrolling is slow enough, that the input queue is never empty, hence
redisplay doesn't get a look in?

> Now, what happens when you release the key?  The input queue is still
> full of unprocessed PageDown keys.  As I explained above, Emacs will
> drain the entire input queue before it enters redisplay; thus the long
> delay you see after releasing the key.

And each such PageDown key will necessitate full font-locking for its
never-to-be-displayed screen.

> You want Emacs to "immediately" display the new buffer position, but
> this is impossible without some code that would quickly scan the input
> queue, analyze what's there, and "understand" that 100 PageDown
> keystrokes can be "optimized" by executing a single scroll-up-command
> with a suitable argument.  IOW, we would need some input preprocessing
> stage that could "optimize" input by replacing a series of commands
> with a single command, and do that cheaply.  We don't have such a
> feature; patches to add it will probably be welcome.

:-).  If my analysis is right, this wouldn't help.  A single mega-scroll
would still be font-locking the entire intermediate buffer region.

-- 
Alan Mackenzie (Nuremberg, Germany).