Re: Compositions and bidi display

[Eli Zaretskii]

> From: Kenichi Handa <address@hidden>
> Cc: address@hidden
> Date: Fri, 30 Apr 2010 15:06:11 +0900
> 
> In the case of "english HEBREW TEXT text" (lowercases are
> l2r characters, upppercases are r2l characters),
> get_next_display_element starts from the first "e" and
> proceeds to the first " " (stage 1), then jumps to the last
> "T" and proceeds back to the first "H" (stage 2), then jumps
> to the last " " and proceeds to the last "t" (stage 3).

This is only the simplest case, with just 2 embedding levels: the base
level of the paragraph, and the (higher) level of the embedded R2L
text.  The general case is much more complex: there could be up to 60
nested levels, and some of them could begin or end at the same buffer
position.  bidi.c handles all this complexity by means of a very
simple algorithm, but that algorithm needs to know a lot about the
characters traversed so far.  I don't think exposing all these
internals to xdisp.c is a good idea.

> Note that composition_compute_stop_pos just finds a stop
> position to check, and the actual checking and composing is
> done by composition_reseat_it which is called by
> CHAR_COMPOSED_P.

Right, but the same is true for the bidi iteration: I need only to
know when to check for composition; the actual composing will be still
done by composition_reseat_it.  I just cannot assume that I always
move linearly forward in the buffer.  Therefore, it is not enough to
have only the next stop position recorded in the iterator.  I need
more information recorded.  What I'm trying to determine in this
thread is what needs to be recorded and how to compute what's needed.
Thanks for helping me.

> > > We may be able to simplify that condition to
> > > "until it reaches a character in the different bidi level
> > > (or chunk)".
> 
> > But that could be very far back.
> 
> Isn't it possible to record where the current bidi-run
> started while you scan a buffer in
> bidi_get_next_char_visually?

See above: it's tricky.  The function in bidi.c that looks for the
beginning and end of a level run relies on almost all the other
functions in bidi.c, and it does that on the fly.  The level edges are
not recorded anywhere, except in an internal cache used to speed up
moving back in the buffer.

> > If MAX_AUTO_COMPOSITION_LOOKBACK is not the right number, then how
> > long can a composition sequence be?
> 
> It is MAX_COMPOSITION_COMPONENTS (16), but here it's not
> relevant.

Why not?  Isn't it true that if none of the 16 characters preceding
the current position can start a composition sequence, then the
current position is not inside a composition sequence?

> > Another idea would be to call composition_compute_stop_pos repeatedly,
> > starting from the last cmp_it->stop_pos, until we find the last
> > stop_pos before the current iterator position, then compute the
> > beginning and end of the composable sequence at that position, and
> > record it in the iterator.  Then we handle the composition when we
> > enter the sequence from either end.
> 
> To move from one composition position to the next, we must
> actually call autocmp_chars and find where the current
> composition ends, then start searching for the next
> composition.  As autocmp_chars calls Lisp and all functions
> to compose characters, it's so inefficient to call it
> repeatedly just to find the last one.

If the buffer or string is full of composed characters, then yes, it
would be a slowdown.  Especially if the number of ``suspect'' stop
positions is much larger than the number of actual composition
sequences.  But what else can be done, given the design of the
compositions that doesn't let us know the sequence length without
actually composing the character?

Thanks.