Re: Native line numbers, final testing

[Alex]

Eli Zaretskii <address@hidden> writes:

>> From: Alex <address@hidden>
>> Cc: address@hidden
>> Date: Fri, 30 Jun 2017 15:15:00 -0600
>> 
>> I noticed another bug. With `display-line-numbers' set to t or 'relative
>> in xdisp.c:
>> 
>> M-g c 2970 RET
>> C-n
>> 
>> Notice that C-n moved back one column. This does not happen with
>> 'visual, nor does it happen when the window is sufficiently small. This
>> is in emacs -Q, and only happens with non-nil line-move-visual.
>
> Such problems (which IMO are minor and quite rare) cannot be helped,
> unfortunately.  They happen near the point where the line numbers will
> soon need one more or one less column.
>
> The underlying issue is how this feature calculates the number of
> columns required for displaying all the numbers in the visible portion
> of a window.  The naïve method would be to perform the window display
> twice, because you only know the largest line number in view after
> actually displaying all the lines.  But that is too slow, and would
> land us right back at the slow operation of linum-mode and friends.
>
> So instead we guess the maximum line number without actually counting
> the lines in the window.  That guess must be conservative, otherwise
> you'll see the text shift to the right in the middle of a window.
> Therefore the guess uses the maximum number of screen lines a window
> could possibly display without resizing, by using the number of rows
> in the underlying display structure called the "glyph matrix".  This
> assumes that (1) all of the glyph-matrix rows will be used (usually
> false, except on TTYs), and (2) that all the shown lines are
> non-continuation lines.
>
> This is why many times you see the number of columns grow before the
> additional column is actually needed by the last line number displayed
> at window's bottom.
>
> The same code, and the same guessing, is also used by various
> functions that simulate display, such as vertical-motion (which is
> called internally by C-n).  Except that the vertical position of point
> in the virtual "window" they consider is taken as zero, because in
> general it cannot be known (the function could be called in the middle
> of a Lisp program that moved point, and the updated vertical position
> will not be determined until the next redisplay cycle).  So the
> resulting number of columns these functions compute could be different
> from what actually is, or will be, on the screen.  And therefore
> column calculations under line-move-visual will err by the same
> amount.
>
> IOW, these issues are a (small, IMO) fine we need to pay for having
> such fast line-number display.
>
> I will keep this issue in mind, in case some idea emerges for how to
> fix them, thanks for pointing it out.

Thanks for explaining, but I disagree on the severity of this bug. It is
relatively rare, but navigating a buffer with C-n/C-p just before the
line 100/1000 isn't exactly uncommon. I believe consistency of C-n/C-p,
at least with the same size window, is quite important. Imagine if
C-f/C-b also went up/down a line if the column number was, e.g., "a bit"
before 10^k, for integers k > 1. That would break users' expectations of
these commands considerably, even if, with the exception of the column
"a bit" before 100, these column positions are rare. This is worsened by
the fact that "a bit" is dependent on the window size, meaning it's
harder to reason about exactly.

One could argue that if you like consistency, then you should set
line-move-visual to nil. I suppose that's true in some cases, but that
doesn't mean that a non-nil value should just ignore consistency if it
can somehow be preserved. Also, visual-line-mode is helpful for some
types of documents (e.g. LaTeX), and it's natural to have
line-move-visual be non-nil in this case. It would be a shame to have to
choose between native display line numbers and visual-line-mode in those
documents.

If no good solution can be found, then I think even a slowdown for
C-n/C-p would be acceptable if it meant a fix for this issue.

I have a couple more questions if that's alright:

It's not intuitive to me why this occurs between lines with the same
display line number width. If vertical-motion uses the same heuristic as
display of line numbers, then why is the column changing between lines
even when the width of line numbers isn't? Is it because it's using
the heuristic with different inputs? If so, can't they be modified to
achieve the same results as the display of line numbers?

Why is it necessary for line numbers to actually affect the vertical
position of characters in the buffer? I suppose it's a bit late to be
asking this question, but the approach from an outside view feels odd. I
don't know what the options were, but it's odd that line numbers aren't
in their own special area like in (n)linum. Does the display engine not
work well with margins?

>> I'm not planning to use 'visual, and I of course don't claim to know
>> anything about this, but I'm curious as to why it works like this.
>> goto-line in either style always goes to the same absolute line number,
>> right? Does the line number display calculation occur after the point is
>> already moved to this line?
>
> Yes.  Line numbers are calculated as part of redisplay, and that
> happens when point already moved.

>> If so, then doesn't the display engine just need to calculate the
>> visual line numbers around this point? If so, then why does a large
>> jump like M-g g 25000 take so much longer comparatively than M-g g
>> 1000? If these assumptions are wrong (they likely are), why are they
>> wrong?
>
> Because the display engine employs a lot of optimizations, to try and
> shortcut expensive redrawing of the entire window, and to comply with
> user expectations.  It doesn't, and cannot, know that buffer position
> 25000 is necessarily outside the window where the command was invoked.
> E.g., you could have some text-folding mode, such as Outline, active
> in the buffer; or you could have lines truncated with only a few very
> long lines between the window-start and position 25000.
>
> So redisplay first tries to reuse the same window-start point as it
> used in the previous cycle, because that's what you as a user would
> expect: if the new point is inside the same window, you don't expect
> the window to scroll.  Deciding whether the previous window-start is
> still usable needs to check whether point will be shown inside the
> window, and that requires to try reaching point from the original
> window-start, in this case position 1, and counting screen lines as we
> do, until we either hit point or the window's bottom.
>
> It turns out that the 'visual' mode of line numbers made this test
> unnecessarily slow, so a simple heuristic was enough to avoid that.
> See the latest branch, where 'visual' and 'relative' should now
> redisplay at the same speed in this case.

Thanks for the explanation and fix. The heuristic does indeed make it a
lot faster.