Re: [emacs-bidi] using fribidi and Pango

[Eli Zaretskii]

> Date: Mon, 5 Dec 2005 01:10:51 -0500 (EST)
> From: Behdad Esfahbod <address@hidden>
> Cc: Emacs Bidirectional Mailing List <address@hidden>
> 
> On Sun, 4 Dec 2005, Eli Zaretskii wrote:
> 
> > > Fortunately a lot of data tables and other stuff can be replaced by
> > > FriBidi's structures, chopping some 500 lines of it I guess.
> >
> > If you mean the tables of character properties, then these will
> > eventually go away in Emacs as well; see the FIXMEs there.
> 
> I was talking from the adopting your code in GNU FriBidi point of
> view.

Yes, me too.  What I meant to say was that, eventually, I will have to
refactor the code so that it uses external tables instead of private
ones.  When that happens, the interface to those tables will present
an API that other packages, like FriBidi, could use to reference their
own tables or data structures.

> > > The rest needs some restructuring.
> >
> > What kind of restructuring, and why is it needed?
> 
> Well, to be honest, the code is far more complex than what I've
> ever seen in any Free Software project.

???  You may wish to have another look at regex.c, for example, it's
no less complex, IMHO.

Not that I consider myself the best software designer out there, but
the structure of the code follows the description in UAX#9 so closely
that I didn't think anyone who has the document handy could have any
difficulty understanding it.  Roughly speaking, there's a function for
each one of the 5 steps of the Unicode algorithm listed in paragraph
3.3 of UAX#9, and then there's a couple of higher-level functions that
know how to find the next visual-order character given the resolved
levels of the current and the next character.  That's about all there
is to it; the rest of the complexity can be mapped directly to what
the Unicode algorithm mandates, and the comments in the code make this
mapping very simple.

> I think I know how complex what you are trying to accomplish is, but
> I don't think the extra complexity of the incremental algorithm is
> worth the effort.

I'm not sure I understand: in what sense is it not ``worth the
effort''?  The decision to code an incremental implementation was made
by discussing possible designs with Gerd Moellmann, the designer and
main implementor of the Emacs 21 redisplay code.  Gerd felt that any
batch-type implementation, and the necessary caching that it brings
with it, would slow down Emacs display for bidi scripts to an
unbearable level.  So writing such an incremental implementation was a
necessity for Emacs, thus clearly ``worth the effort''.

> As one can guess, debugging it is a nightmare

Actually, I discovered that it isn't very hard to debug the code.  The
function that dumps the resolved levels makes it easy to spot
problems, and the calls to `abort' where unexpected things happen
reveals problems very early.

> Ok, I'm not an Emacs user/developer at all, but I really don't
> understand why Emacs should be called a heavyweight display
> monster.

Because each buffer position can have very complex data structured
associated with them that affect various aspects of display in
arbitrary complex ways.  Just to scratch the surface: Text properties
and overlays can specify that, instead of displaying the current
character in the default font and color, Emacs should display some
other text, or use some other font/color, or display an image, or
evaluate arbitrary Lisp expression and use the result for display, or
make the text invisible, etc.  Each character can have entries in
various tables which modify the glyph (or a string of glyphs) that
need to be produced to display that character.  Every possible type of
glyph consults many user options and other Lisp-level structures and
functions to determine how the glyphs need to be produced and laid
out.

In other words, the layout engine has so many hooks into the
surrounding Emacs features that I doubt any external layout
widget/library could do the same job, and do it efficiently.

> Anyway, the only reason I spoke up was to tell you that your
> contributions to whatever project I'm involved in is always
> welcome and appreciated.

Thanks, I had no doubt in that.