Re: Assertion failure in set_iterator_to_next

[Eli Zaretskii]

> From: Stefan Monnier <address@hidden>
> Cc: address@hidden
> Date: Sun, 11 Apr 2010 11:02:22 -0400
> 
> > No, it->len should be the length of the multibyte sequence of the
> > character at IT_CHARPOS (*it).  This is how the basic iteration works,
> > and if the length is incorrect, we will get garbled display, in
> > anything but pure 7-bit ASCII buffers.
> 
> OK, good.  Can you update/complete dispextern.h correspondingly?

Will do.

> So the problem now is "why is it->len ==3 rather than ==1"?

I think it's because it was not yet updated.  It is updated only in
the next_element_from_* functions.  E.g., for a character from a
buffer, we have this in next_element_from_buffer:

      /* Get the next character, maybe multibyte.  */
      p = BYTE_POS_ADDR (IT_BYTEPOS (*it));
      if (it->multibyte_p && !ASCII_BYTE_P (*p))
        it->c = STRING_CHAR_AND_LENGTH (p, it->len);     <<<<<<<<<
      else
        it->c = *p, it->len = 1;                         <<<<<<<<<

where I marked the 2 lines that update it->len.  These functions are
again called only from get_next_display_element, which was not yet
called at the point where you have the faulting xassert.

IOW, it->len is correct _before_ we advance to the next character, and
it holds the length of the multibyte sequence of the character we
already consumed, the one we are stepping over by incrementing bytepos
by it->len.

However, all this does not explain at all why you get the assertion
failure, it just explains why neither the value of it->c nor it->len
are relevant to the failure.  All this notwithstanding, it is still
true that `IT_BYTEPOS (*it) == CHAR_TO_BYTE (IT_CHARPOS (*it))', I
think.

Since you are saying this is an ASCII buffer, are charpos and bytepos
at least identical?  If they are, could it be that CHAR_TO_BYTE has a
bug?  If they are not identical, how large is the discrepancy?