Re: Long lines and bidi [Was: Re: bug#13623: ...]

[Dmitry Antipov]

On 02/08/2013 06:07 PM, Eli Zaretskii wrote:

> Profile alone is not enough.  Please tell how did you "scroll",
> exactly (which commands did you use), and please also show the
> absolute times it took to perform each command.

(defun scroll-both ()
  (interactive)
  (let ((start (float-time)))
    (progn
      (dotimes (n 100) (progn (scroll-up) (redisplay)))
      (goto-char (point-max))
      (dotimes (n 100) (progn (scroll-down) (redisplay)))
      (message "Elapsed %f seconds" (- (float-time) start)))))

With bidi, ~600 second elapsed, and:

    25.18%        emacs  emacs                          [.] scan_buffer
     7.04%        emacs  emacs                          [.] bidi_resolve_weak
     6.47%        emacs  emacs                          [.] 
get_next_display_element
     6.37%        emacs  emacs                          [.] 
bidi_level_of_next_char
     5.14%        emacs  libc-2.16.so                   [.] __memcpy_ssse3_back
     5.05%        emacs  emacs                          [.] 
move_it_in_display_line_to
     4.94%        emacs  emacs                          [.] x_produce_glyphs
     4.84%        emacs  libXft.so.2.3.1                [.] XftCharIndex
     3.72%        emacs  emacs                          [.] 
bidi_move_to_visually_next
     3.70%        emacs  emacs                          [.] 
next_element_from_buffer
     2.90%        emacs  libXft.so.2.3.1                [.] XftGlyphExtents
     2.05%        emacs  emacs                          [.] bidi_fetch_char
     2.02%        emacs  emacs                          [.] 
lookup_glyphless_char_display
     2.01%        emacs  emacs                          [.] bidi_resolve_neutral
     1.76%        emacs  emacs                          [.] 
bidi_cache_iterator_state
     1.70%        emacs  emacs                          [.] bidi_get_type
     1.51%        emacs  emacs                          [.] 
bidi_resolve_explicit_1
     1.18%        emacs  libXft.so.2.3.1                [.] XftFontCheckGlyph
     1.12%        emacs  emacs                          [.] xftfont_encode_char
     1.01%        emacs  emacs                          [.] xftfont_text_extents

Without bidi, ~230 seconds elapsed, and:

    21.36%        emacs  emacs                          [.] x_produce_glyphs
    17.92%        emacs  emacs                          [.] 
get_next_display_element
    15.07%        emacs  emacs                          [.] 
move_it_in_display_line_to
     8.37%        emacs  emacs                          [.] 
next_element_from_buffer
     8.34%        emacs  libXft.so.2.3.1                [.] XftCharIndex
     6.12%        emacs  emacs                          [.] 
lookup_glyphless_char_display
     4.21%        emacs  libXft.so.2.3.1                [.] XftGlyphExtents
     3.07%        emacs  emacs                          [.] xftfont_encode_char
     2.68%        emacs  emacs                          [.] xftfont_text_extents
     1.87%        emacs  emacs                          [.] get_per_char_metric
     1.53%        emacs  libXft.so.2.3.1                [.] XftFontCheckGlyph
     1.49%        emacs  emacs                          [.] 
composition_compute_stop_pos
     1.35%        emacs  emacs                          [.] set_iterator_to_next

cache-long-line-scans is nil in both cases.

I suspect that scroll should be direction-agnostic in theory; but both profiled
runs shows that scroll-down is much, much slower than scroll-up (that's why
elapsed time is so huge in both cases).

> What was in the file?  bidi_resolve_weak high on the profile hints
> that it was full of punctuation or digits or banks, which is not
> really an interesting case.

Your guess is correct; but I suspect that an average text in human language
contains less punctuations, digits and blanks than the C source code of the
same size :-).

> Ah, _that_ red herring...  Why is that the first question?  What were
> the times with and without bidi-display-reordering in this file?  In
> my testing, the display engine performs awfully slow in both cases, so
> even though turning off reordering makes it faster, it is still so
> terribly slow that the problem is not going to be solved by that.
>
> As to your question: how can we know what characters are or aren't in
> the buffer without scanning it?  And scanning the buffer is exactly
> what bidi.c does.

Hm... insert-file-contents tries to detect encoding by looking at first 1K
and last 3K of the file. Why the similar approach isn't applicable to bidi?

Dmitry